Example-based machine translation by analogy is an alternative approach to machine translation. Its principle is relatively simple, but the absolute number of analogies between sentences contained in the corpus is crucial for the overall quality of translation. The relative number of analogies is called the analogical density. The goal of this paper is to measure the analogical density of diferent aligned corpora. To this end, we extract analogies between sentences. Now, we use parse trees to represent sentences on the level of syntax. We report analogical densities for five diferent languages in an aligned multilingual corpus extracted from the Tatoeba resource, at the level of characters, words or parse trees.
: :: : ↕ ↕ ↕
↕ ′ : ′ :: ′ : ′ (1) I like apples.
: Ipledso.n’t like ap- :: I speak
Swedish.
: I don’t
Swedish.
speak
The number of analogies that exist in a given corpus is crucial for EBMT by analogy. Our objective in the present work is to estimate the number of analogies similar to the one shown in Figure 1, for various language pairs. Now, analogies can be extracted at various levels: surface form or syntax. To extract analogies automatically, we use vector representations of sentences based on the occurrence of characters, tokens, or branches in parse trees. We then count the number of extracted analogies and can compute the analogical density of the corpus. Although we do not conduct experiments in this paper, our intuition is that a higher number of analogies will lead to better translations in an EBMT system by analogy.
Formal analogies do not take into account the meaning or the syntax of sentences. Instead, the
surface form, i.e., characters or words, are only taken into account. [
In an analogy at the semantic level, the meaning attached to the strings is considered. For
instance, king : queen :: man : woman is a classic example of semantic analogy [
In the present paper, we concentrate on analogies between sentences. In [
The boy speaks : The girl goes to :: The actor spoke : The actress went Thai. Thailand. Chinese. to China.
I talk to him. : I talked to him. :: I go to school. : I goed to school.
NP
S
VP
been described. It relied on the use of analogy. Similarly, an example of an analogy between
syntactic trees is shown in [
[
We propose to work on analogy at the level of syntax. Figure 2 is another example of a syntactic analogy between sentences. There, the sentences do not acceptedly create an analogy on the level of form or meaning, but they definitely make an analogy at the syntactic level: exchange of personal pronoun (PRP) with proper noun (NNP). Notice that, for the analogy to hold, the terminals (the words in the sentences) which should appear on the leaves in the parse trees are not considered.
While past studies concentrated on analogies on the formal level, the originality of this paper is to extract and count analogies between sentences on the level of syntax using parse trees. To this end, we develop two components. The first component computes vector representations. A sentence is represented by a feature vector counting the number of occurrences of all branches in any parse tree of a sentence from the corpus considered. The second component computes VBD
VBD
VBD
VBD PRP . : NNP . :: PRP . : NNP the ratio between these vectors of features on two given trees. The ratio between sentences is simply defined as the diference between their feature vectors.
In computational linguistics, a parse tree is a tree that represents the syntactic structure of a
sentence [
A sentence can be represented by a feature vecto→r− by counting the number of occurrences for all the branches found in its parse tree . In Formula (2), the notation ||branch stands for the number of times a branch appears in the parse tree of sentence .
⎛ || → ⎞ →− = ⎜⎜⎜⎝ || ...→ ⎟⎟⎟⎠
|| →.
To extract analogies at the level of syntax, we calculate the ratio between trees. Formula (3) defines the ratio between sentences and as the diference between their vectors of syntactic features derived from their parse trees and .
: ≜ − = ⎜⎜ || → − | | → ⎟ →− →− ⎜⎝ ... ⎟⎠⎟ ⎛ || → − | | → ⎞
|| →. − | | →.
An analogy A : B :: C : D is satisfied by checking the equality of ratios. Formula (4) defines it.
For the computation of ratios between vectors and for checking for equality of ratios, we rely
1spaCy: https://spacy.io/
on the Python library Nlg2 [
: :: :
Δ ⇐⇒ →− − →− =→− − →−
An analogical cluster is defined as a set of pairs of sentences with exactly the same ratio [
. :
Δ ⇐⇒ ∀(, ) ∈ {1, . . . , }2, : :: : (4) (5)
We use the Tatoeba3 corpus. It is a collection of sentences in more than 100 languages. Here, we use five language parts from the Tatoeba corpus: English, French, German, Polish and Finnish. The sentences we used are aligned across all five languages, they are parallel sentences that correspond to each other. Table 2 gives some statistics on this corpus. For each language, we have around eight thousand sentences. English has the lowest number of types and Finnish has the largest one among the five languages. Hapaxes are words that appear only once in a corpus. Here, we observe that English has the smallest number of hapaxes with less than 60% while Finnish has the highest percentage with over than 70%. We verify again that languages with higher morphological richness tend to have a higher number of types and hapaxes. In interest to us is the conjecture that we should extract more analogies from a language with a higher Type-Token Ratio (TTR) since type-token ratio measures lexical richness.
To evaluate the number of analogies between sentences contained in a corpus, two metrics used
in [
Formula (6) defines analogical density as the ratio of the number of actual analogies and
4. If the total number of sentences in the corpus is , 4 is the number of possibilities of
2Nlg: http://lepage-lab.ips.waseda.ac.jp/en/projects/kakenhi-15k00317/
3Tatoeba: https://tatoeba.org/
Language
iflling in the analogy pattern with any four sentences (with possible repetition) from the corpus.
As there are 8 equivalent forms of analogies [
Formula (7) calculates the proportion of sentences appearing in analogies by dividing the number of sentences appearing in at least one analogy (s_nlg) by the total number of sentences in the corpus (). This makes a percentage.
We carry out experiments on the extraction of analogies between sentences both on the level of surface form and syntax. On the level of form, each sentence is tokenised using two diferent tokenisation schemes: character or word. On the level of syntax, we extract sentence analogies from a corpus by using parse trees, as described in Section 3. We also concatenate formal feature vectors with syntax feature vectors to combine the two levels. We do not conduct experiments using char and word features at the same time, because they both work on the formal level.
Language languages, followed by Polish and French. Except for char, we observe that German always has the highest number of analogical clusters (except for char where it is second).
In addition, we draw the distribution of the number of analogical clusters with the same size extracted from syntactic features for our five languages in Figure 4. Although the numbers of extracted analogical clusters with the same size vary across languages, the overall trend is consistent.
The analogical density of the corpus is presented in Table 4. It indicates how many analogies can be extracted in the vfie diferent languages and how many sentences can be covered by these extracted analogies.
We observe that the number of analogies extracted on the level of syntax is thousands times more than on the level of form. Basically, on the level of syntax, Finnish has the highest
Size analogical density and the sentences that appear in analogy account for 88 percent of the whole corpus.
Figure 5 shows the number of features per sentence appearing in the extracted analogies on diferent levels. It is obvious that compared to the formal level, we extract more analogies on the syntactic level, given the smaller vector representations using parse trees. . . .
: :: :
PRP I left the door open .
He left the door open .
In Table 5, we list some example results of sentence analogies that we extracted from the corpus in the combination of formal and syntactic features. Figure 6 plots the syntactic structure behind the first English example in Table 5.
An analogical grid is a matrix where any four terms picked out from any two rows and any two
columns is an analogy [
We won. : Tom won. : You won.
We survived. : Tom survived. : You survived.
: Tom drank too much. : You drank too much.
We volunteered. : Tom volunteered. : an analogical grid is defined as the product of its number of rows by its number of columns. As shown in Figure 7, an analogical grid may have empty cells. Thus, we can also characterise an analogical grid by the number of non-empty cells in it. This is its saturation. It is the ratio between the number of non-empty cells and the size of the grid.
11 : 21 : · · · : 1 12 : 22 : · · · : 2
... ... ... 1 : 2 : · · · :
Δ ⇐⇒ ∀(, ) ∈ {1, . . . , }2, ∀(, ) ∈ {1, . . . , }2, : :: :
We have extracted analogical grids on the level of syntax combined with character features or word features. By extracting the analogical grids, we hope to get a more compact view of how sentences are related to each other by analogies. Based on Table 6, we observe that English has the highest number of analogical grids but also the smallest average size of analogical grids. German has the largest average size of analogical grids. The average saturation of the extracted analogical grids is all around 99 % which means the analogical grids extracted from our corpus are very dense.
For any language pair from the five languages in the aligned corpora, we can extract bilingual analogies by taking monolingual analogies where sentences correspond by translation. This kind of data, i.e., bilingual analogies, can then be exploited in an EBMT system by analogy. (8)
We proposed to extract analogies between sentences based on their syntactic structure. Experiments were carried out using Universal Dependency parse trees that allow us to compare across ifve diferent European languages. The parse trees were converted into feature vectors, the features of which were the types of branches, from which we removed the lexical information. We measured the analogical density at the syntactic level and crossed with the results at the character or word levels.
We found that the number of analogies extracted on the syntactic level is hundreds or thousands times larger than the one on the formal level, which leads to a thousand times higher analogical density. We already started extracting analogical grids to have a more compact view of how sentences are related to each other.
In this paper, we used the number of occurrences of branches in dependency representations as features to get a vector representation of sentences. Similar work could be carried out with constituency representations, if constituency parsers comparable across languages would be available. The ultimate goal of the work presented here, is to not only to extract monolingual analogies, but bilingual analogies between sentences, because they can be used by an EBMT system by analogy.