Syntactic Translation Patterns from a Parallel Treebank

Abstract

The goal of the presented parallel phrase extraction algorithm is to provide rich and robust set of translation syntactic patterns. To make this approach feasible, we consider the phrase-to-phrase alignments of a bilingual treebank annotated with syntactic constituents. For the intended purpose, the extracted phrasal nodes are encoded by the syntactical information of their components, highlighting some special constructs such as the functional words.

Keywords

• Parallel syntactic patterns
• phrase-based translation

1. INTRODUCTION

Parallel corpora can be used in order to generate extremely valuable linguistic knowledge such as they can support automatic identification of segments of texts that represent reciprocal translations [ 13 ]. Two segments of texts from a bitext (parallel corpora) which represent reciprocal translations make a translation unit [ 13 ]. The translation units that correspond to syntactic phrases can be used to generate other sentences in the target language of a Machine Translation system: instead of generating translation of individual words in the source language, generate translations of the phrases and assemble the final translation by a permutation of these [ 14 ].

Methods for Machine Translation (MT) have increasingly leveraged not only the formal machinery of syntax but also linguistic tree structures of either the source language, the target language or both. Phrase based statistical MT (PBSMT) techniques for extracting phrases although not syntactically motivated, enjoy a very high coverage [ 1 ]. Basic PB-SMT systems work with phrase pairs that are consistent with the word alignment: the words of a phrase are contiguous strings consisting of words aligned to each other and not to words outside [ 8 ].

Machine translation based on syntactic trees has been extensively studied in the last years due to the general need of improving the performance of the state-of-the-art PB-SMT [ 2 ].

Alignment of the parse trees can offer structural alignment between two parallel sentences, more precisely, can help an experiment for testing the feasibility of the automatic crosslingual transfer of syntactic constituents. Broadly speaking, a transfer component is a system of rules that relate words and structures in one language to words and structures of another language (the target language).

Traditionally, phrases are taken to be syntactic constituents of a sentence. Even if not all the words between two phrases are aligned, the phrases can still align very well [ 11 ]. By aligning the inner nodes of two parallel parse trees, the phrases represented by these nodes are put in correspondence as the subtrees of the syntactic analysis encode the structure of the represented syntactic phrases.

Such techniques have shown that starting with large syntactic phrase tables and preferring syntactic phrases when overlapping with non-syntactic ones allows the learning of ”translation knowledge”. They show improvements in decoding speeds and also improvement in translation quality that results from the precision of these syntax motivated phrases [ 1 ].

Most of the phrases identified in the parse trees are expected to be translated without interleaving with other phrases or words. In general, noun phrases tend to obey the above rule in a much greater degree. At the opposite corner, the verb phrases usually suffer modifications in structure during translation caused by the adjunct movement [ 4 ].

The goal of the presented algorithm for extracting parallel syntactic patterns from a bilingual treebank is to generate a set of good-quality translation patterns intended to be learned by a statistical Syntax-based Machine Translation.

The presented parallel syntactic sequences were extracted from a treebank with syntactic constituents, an EnglishRomanian Treebank [ 5 ]. The treebank was built upon a parallel English-Romanian corpus word-aligned and annotated at the morphological and syntactic level. The syntactic trees of the Romanian texts are generated based on the syntactic phrases of the English parallel texts automatically obtained by means of a syntactic parser, the Standford Parser [ 12 ]. The Romanian trees generation mechanism reuses and adjusts existing tools and algorithms for cross-lingual transfer of syntactic constituents and syntactic trees alignment.

The treebank was constructed upon 1420 sentences from an English-Romanian parallel corpus developed at A.I.Cuza University of Ia¸si by the Natural Language Processing Group of Faculty of Computer Science. The corpus is XML encoded obeying a simplified form of the XCES standard [ 10 ]. For the bilingual corpus construction, the English and Romanian parts of the Acquis-Communitaire1 corpus were used.

All the words of this English-Romanian corpus are annotated with lemmas, morphosyntactic information (gender, number, person and case) and Part of Speech markers. The tagsets used to annotate the words come from MULTEXTEast morphosyntactic specifications (the latest version of these specifications is given in [ 7 ]).

PARALLEL PATTERNS WITH SYNTAC

TIC CONSTITUENTS

Following the method of Galley described in [ 9 ], the phrase extraction process is supported by the parallel parse trees of the constructed English-Romanian treebank. For each alignment between inner nodes of the syntactic trees, the descendents of aligned nodes are examined. According to the purpose for which the syntactic sequences are extracted, in the list of descendents, some specific words or constructions of certain structure can be highlighted.

For the presented article, the syntactic sequences are intended to provide information about the manner in which the functional words can affect translation. For this reason the functional words are given in the complete word-form accompanied with complete information about their morphosyntactic properties.

In any syntactic structure we can identify two major categories of words: content words which describe objects, entities, properties, relationships or events and which are syntactically represented by nouns, adjectives, verbs and adverbs and functional words that help putting words together in a correct structural sentence form. Also, the functional words can tell how the other components of the sentence are related to each other. The functional words can be determiners, quantifiers, prepositions or connectives. 1Acquis Communitaire corpus contains about 12,000 Romanian documents and 6,256 parallel English-Romanian documents [ 6 ].

The span of a node n of a syntactic tree is taken to be the subset of nodes that are reachable from n [ 9 ]. In a bottomup fashion, the algorithm for extracting parallel syntactic patterns “visits” each English syntactic tree and expands all its inner nodes that are aligned with at least one node from the Romanian parallel syntactic tree. The spans of the aligned English and Romanian phrasal nodes are taken to be the parallel syntactic patterns of our study and therefore are stored in a database (see Section 2.2). The method is quick and easy enough to be used on large-scale data sets.

Here are some examples of the importance of parallel syntactic patterns from automatically learned translation rules point of view: • simple lexical patterns for translating special words such as, functional words can be treated as examples of patterns in which optional modifiers are inserted • patterns in which we found ”lexical holes” determined by existence of one-to-zero alignment mapping between the words/tokens of the parallel sequences. For example, English noun phrases that contain the word ”of” as separator. • analyzing large sets of the parallel patterns, we can identify the“part of speech afinities”; it is usually known that translated words tend to keep their part of speech but when this is not the case, the resulted part-ofspeech for the translation is not random.

From the English-Romanian Parallel Treebank with syntactic Constituents, 2120 English-Romanian syntactic patterns with functional word were extracted. The representation in which the patterns are stored can provide good enough descriptions of the domain of locality for the functional words.

2.1 Representation Formalism

The representation for the English syntactic sequences with functional words is an ordered sequence of elements given in the following form:

[ { Phrasal−Tag }∗ Pos−Tag/FW { Phrasal−Tag}∗ ] where by F W we note a functional word. (a) [ IN/at, NP ] (b) [ IN/by, NP ] (c) [ IN/for, NP ]

By parsing the English sentences with Stanford Parser, PENN Treebank parse trees were generated. As a direct consequence, the English texts are annotated with PENN Phrasal tags as this is the tagging standard used by Stanford Parser. In this annotation formalism, the functional words for the English texts can be considered as sentence tokens that in PENN POS tagset formalism have one of the following tags: CC (coordinating conjunction), DT (determiner) , IN (preposition/ subordinating conjunction), MD (modal), PRP (personal pronoun), PP$ (possessive pronoun), RP (particle), TO (word to), WDT (wh-determiner), WP (whpronoun), WP$ (possessive wh-pronoun), WRB (wh-adverb).

Here are some examples of English syntactic patterns: • [NP, PRP, CC/and, NP] the syntactic phrase having this span is made of two noun phrases (NP) linked by a personal pronoun (PRP) and a functional word, the coordinating conjunction and (in this specific order). The two syntactic phrases NP are not expanded because each of them has its own alignment, and thus, their structure is given in other parallel syntactic sequence. • [RB, JJ, CC/and, JJ] the syntactic phrase having this structure contains two adjectives (JJ) linked by a functional word (the conjunction and ) and preceded by an adverb (RB).

Following the same representations, the corresponding Romanian syntactic sequences are encoded in a similar format.

The Romanian syntactic trees of the English-Romanian Treebank were automatically constructed by means of a bottom-up tree generation algorithm guided by the word alignments of the corpus ([ 5 ]). As a consequence, the annotations for the Romanian words preserve the MULTEXTEAST words specifications of the corpus as these data include enough morphosyntactic details needed in any syntactic study, while for labeling the phrasal constituents, the PENN Treebank Phrasal tags are used.

As a direct consequence, the Romanian functional words are those tokens/words that in MULTEXT-EAST Tagset formalism have MSD tags with the following prefixes: P− (pronoun) such as Pd− (demonstrative pronoun), Ps− (possessive pronoun), Px− (reflexive pronoun), D− (determiner), T− (article), S− (adposition), C− (conjunction), Q− (particle).

The representation for the Romanian syntactic sequences with functional words is an ordered sequence of elements given in the following form:

[ { Phrasal−Tag }∗ MSD Tag/FW { Phrasal−Tag}∗ ] where by F W we note a functional word and by M SD we note the morphosyntactic descriptions encoded in MULTEXTEast morphosyntactic specifications.

Here are some examples of Romanian syntactic patterns: • [Di3-po—e/altor, NP] the syntactic phrase given by this sequence contains a determiner (a MSD tag starting with D) followed by a noun phrase • [VP, Crssp/¸si, Tsfs/a, NP] the syntactic phrase whose structure is encoded in this pattern is made of a verb phrase (VP) and a noun phrase (NP) liked by a conjunction (a C− MSD tag) and an article (a T− MSD tag). 2.2

Linguistic Resource with Syntactic Patterns

Each resulted parallel sequence is stored into a database record with four fields (see Figure 1). The SynP hrase En field stores the span of an English syntactic phrase, while in the SynP hrase RO field the span of the aligned Romanian syntactic phrase is given. The last two fields include the PENN syntactic subtrees rooted at the aligned syntactic phrases.

Indeed, T reebank EN gives the bracket representation for the subtree rooted at the English phrase while T reebank RO is the subtree corresponding to the Romanian phrase. Examples of some records of this linguistic resource are listed in Table 1.

Certain statistics about the translation of a particular English syntactic sequence into Romanian language can be easily obtained upon the constructed database table with the described information. [IN/as, NP] [Rw/caˆt, mai/Rp, ADJP] [PP [IN Rsp/as] [NP [NP Afp/strict] [ADJP [PP [Rw 14/caˆt] [Rp 15/mai] [ADJP [RB Cs/as] [JJ Afp/possible]]]] [Afpfp-n 16/stricte] [ADJP [Rgp 17/posibil]]]] [IN/at, NP] [Spsa/la, NP] [PP [IN Sp/at] [NP [NP [DT Dd/the] [NN [NP [Spsa 1/la] [NP [NP [Ncfsry Ncns/end]] [PP [IN Sp/of] [NP [DT Dd/the] 2/ˆıncheierea]] [NP [Ncmsoy 3/exerci¸ti[JJ Afp/financial] [NN Ncns/year]]]]] ului] [ADJP [Afpms-n 4/financiar]]]]]

From the statistics illustrated in Figure 2, one can observe that the translation in Romanian for the English syntactic pattern [IN/at, NP] do not change the order between the noun phrase and the preceding preposition and replace the preposition “at” with the Romanian preposition “la”.

The preposition “by” from the English pattern [IN/by, NP] is equally translated with Romanian prepositions “de” and with the Romanian prepositional collocation “de−c˘atre”, while the preposition “for” from the English sequence [IN/for, NP] is translated with the Romanian preposition “pentru”.

CONCLUSIONS

Statistical Machine Translation systems that use syntactical information in the translation process must be trained with syntactic patterns that correspond to reciprocal translations in the languages of the MT system. Such training can help the translation not only with the structural differences between the translations but also with the re-ordering problems at the target sentence words [ 3 ].

Even if the lexical coverage of the used corpus, the Acquis Communitaire corpus, is not representative, a MT system can still benefit from the translations similar in structure and semantics that exist between the parallel sentences of the corpus.

Also the meanings of some special words, such as functional words, can be easily explored by analysing the changes during the translation suffered by syntactic patterns consisting of this kind of words. In the way it is constructed now, the resource focuses on the importance the functional words have in a translation process. But the syntactic patterns can be generated in order to highlight other constructions, for example, the polylexicals units of a natural language phrase.

As a future work, we intend to enlarge the size of the bilingual treebank in order to permit generation of a larger set of parallel syntactic patterns.

ACKNOWLEDGMENTS

The author M. Colhon has been funded for this research by the strategic grant POSDRU/89/1.5/S/61968, Project ID 61986 (2009), co-financed by the European Social Fund within the Sectorial Operational Program Human Resources Development 2007-2013.

References

[1] V. Ambati , A. Lavie , and J. Carbonell. Extraction of syntactic translation models from parallel data using syntax from source and target languages . In MT Summit XII , 2009 .

[2] J. G. Arau`jo and H. M. Caseli . Alignment of portuguese-english syntactic trees using part-of-speech filters . http://www.cs.famaf.unc.edu.ar/~laura/ nlpw/nlpw/papers/Araujo_Caseli.pdf. Online; accessed 19-June-2012.

[3] A. Ceau ¸su. Rich morfo-syntactic description for factored machine translation with highly inflected languages as target . In Workshop on Machine Translation and Morphologically-rich Languages , University of Haifa, 2011 .

[4] M. Colhon . A contrastive study of syntactic constituents in english and romanian texts . In Proc. of the Workshop “Language Resources and Tools with Industrial Applications” , pages 11 - 20 , 2011 .

[5] M. Colhon . Language engineering for syntactic knowledge transfer (submitted) . Computer Science and Information Systems Journal , 2012 .

[6] D. Cristea and C. For˘ascu. Linguistic resources and technologies for romanian language . Computer Science Journal of Moldova , 14 ( 1 ( 40 )), 2006 .

[7] T. Erjavec . Multext-east version 4: Multilingual morphosyntactic specifications, lexicons and corpora . In Proc. of LREC'10. ELRA , 2010 .

[8] M. K. G. Wenniger and K. Sima'an. A toolkit for visualizing the coherence of tree-based reordering with word-alignments . In Proc. of the 5h MT-Marathon , pages 97 - 104 , 2010 .

[9] M. Galley , M. Hopkins , K. Knight , and D. Marcu . What's in a translation rule? In Proc. of HLT-NAACL 2004 , pages 273 - 280 . ACL, Boston, USA, 2004 .

[10] N. Ide , P. Bonhomme , and L. Romary . Xces: An xml-based encoding standard for linguistic corpora . In Proc. of the 2nd LREC , Paris: ELRA, 2000 .

[11] R. Ion , R. Ceau¸su, and D. Tufi¸s. Dependency-based phrase alignment . In Proc. of the 5th LREC , pages 1290 - 1293 , 2006 .

[12] D. Klein and C. D. Manning . Accurate unlexicalized parsing . In Proc. of the 41st Annual Meeting of ACL , pages 423 - 430 , 2003 .

[13] D. Tufi ¸s and R. Ion . Parallel corpora, alignment technologies and further prospects in multilingual resources and technology infrastructure . In Proc. of the 4th SPED , 2007 .

[14] K. Yamada and K. Knight . A syntax-based statistical translation model . In Proc. of ACL , pages 523 - 530 , 2001 .