The Greek and Latin classics, like many other ancient texts, have been widely translated into a variety of languages over the past two millennia. Although many digital editions and libraries contain one or two translations for a given text, about one hundred translations of Itlhiaed and twenty of Herodotus, for example, exist in English alone. Aligning the corpus of classical texts and translations at the sentence and word level would provide a valuable resource for studying translation theory, digital humanities, and natural language processing (NLP). Precise and faithful sentence alignment via computational methods, however, remains a challenging problem. Current alignment methods tend to have poor coverage and recall since their primary aim is to extract single sentence pairs for training machine translation systems. This paper evaluates and examines the limits of such state-of-the-art models for cross-language sentence embedding and alignment of ancient Greek and Latin texts with translations into English, French, German, and Persian. We release evaluation data for PlatoC'rsito, manually annotated at the word and sentence level, and larger test datasets based on coarser structural metadata for Thucydides (Greek) and Lucretius (Latin). Testing LASER and LaBSE for sentence embedding and nearest-neighbor retrieval and Vecalign for sentence alignment, we found best results using LaBSE-Vecalign. LaBSE worked surprisingly well on ancient Greek, most probably because it had been merged with modern Greek data in its training. Both LASER-Vecalign and LaBSE-Vecalign did best when there were many ground-truth one-to-one alignments between source and target sentences, and when the order of sentences in the source was preserved in the translation. However, these conditions are o昀琀en not present in the kinds of literary and free translation we wish to study, nor in editions with multiple translations, extensive commentary, or other paratext. We perform book-level and chapter-level error analysis to inform the development of a so昀琀ware pipeline that can be deployed on the vast corpus of translations of ancient texts.
Texts from the corpus of ancient Greek and Latin have been translated multiple times into many languages. Having access to sentence-level links between original texts and their translations would be useful to students and researchers alike. Without access to living speakers, being able to easily consult multiple translations at a granular level while reading an ancient text would enrich the reader’s understanding of the origin7a,l6[]. This would also open up access to non-experts in the source language, ancient or moder8n].[ Moreover, such a dataset would represent a valuable resource to the machine learning community. Ancient texts have been translated many times into multiple languages and span di昀erent styles, content, and contexts, a variety that makes for “an excellent challenge for NLP2”].[ They also provide ample source material for a large and varied dataset of multilingual parallel sentences. Finally, sentence alignment—the process of automatically matching corresponding sentences in a source text and its translation—is crucial for 昀椀ner-grained computational analysis of translations. This is because a large number of models and methods for computationally processing multilingual data assume access to parallel data at the level of sentences or small chunks of text.
This paper presents our work evaluating approaches to sentence alignment and their ability
to work with pre-modern and literary texts. In so doing, we aim to shed light on the features of
this data that can present challenges and the types of errors that are likely to occur. In addition
to the challenges associated with processing ancient languages with limited resources, we also
provide evidence for additional sources of error associated with sentence alignment of ancient
texts and their translationsn:oisiness in the translations due to the presence of substantial
paratext including footnotes, commentaries, multiple translations, and chunks of source text itself.
We also present the pipeline that achieved the best results, LaBSE - Vecalig1n1[
CPWrEooUrckResehdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g CEUR Workshop ProceedingsC(EUR-WS.org) 1As described in §4.2.1, Farnoosh Shamsian supervised the students performing these alignments and producing the Persian translation as part of a project at the University of Leipzig. the English translation of Lucretius contain substantial paratext; all other translations and the original ancient Greek and Latin texts do not. In the next phase of our work, we will apply LaBSE - Vecalign to the database of 1,526 translations of Greco-Roman texts compiled by the Open Greek and Latin Project (OGL2) and share their sentence-level alignments.
Sentence alignment has not received much attention as an end in itself, with model2s5[
Word alignment has received substantial coverage by the natural language processing
community [
We describe our pipeline to align parallel sentences across source texts and their translations, which consists of the following steps: a) segment the texts into sentence-level chunks via automatic heuristics and language-speci昀椀c sentence segmentation tools like stanza4[], b) compute low-dimensional representations (embeddings) for the segmented sentences that are speci昀椀cally designed to be informative about semantic similarity between sentences in multiple languages, c) use the embeddings across the sentences from a source text and its translation to guide prediction of sentence alignments.
We tested two models for obtaining sentence representations: LASER1][and LaBSE [
Both LASER and LaBSE leverage vast amounts of sentence-level parallel data to map
sentences that are translations of each other to a shared low-dimensional manifold such that their
representations are close to each other. LASER is an encoder-decoder LSTM model trained
using a translation task with publicly available parallel data across numerous language pairs. The
encoder is shared across the languages and the datasets. In contrast, LaBSE is a BERT-based
[
We tested both LASER and LaBSE without further 昀椀ne-tuning. Surprisingly, our experiments suggested that LaBSE was able to generalize well tuonsupported languages like Ancient Greek and Latin that we focus on in this paper. In addition, as we suggest in the Discussion section, we believe that a larger share of errors, especially when working with noisy, ancient texts, would be mitigated by improvements to the alignment algorithm. We note this in our Conclusion as a possible direction for future work.
For sentence alignment, we relied on Vecalign25[], which remains the state-of-the-art model though it was published in 2019. Vecalign computes a昀케nity scores between sentences or groups of sentences from a source and target text. The algorithm takes in sentence embeddings as input and then uses these embeddings to assess the similarity of sentences. Then, it reduces the problem of alignment to enumeration over all possible pairs of groups of sentences across the source and target texts. This process yields highest scoring pairs, resulting in either one-to-one, one-to-many, many-to-one, or many-to-many aligned sentence pairs. This enumeration is exponentially expensive in terms of the size of the bitext (pair of source and target texts). Therefore, a dynamic program is used to perform it e昀케ciently. Further approximations are made to reduce the runtime by incorporating inductive biases and modeling assumptions, such as the largely monotonic nature of sentence alignment across the bitext. Another source of drastic reduction in runtime is a coarse-to-昀椀ne approach employed for alignment, which prunes the search space of sentence pairs aggressively by making severe contiguity and monotonicity assumptions. These approximations and assumptions reduce the runtime to being asymptotically linear in terms of the size of the corpus. However, as we show in our experiments, these assumptions might not necessarily hold for our task. The texts we are interested in exhibit signi昀椀cant non-monotonicity, discontiguousness, and noise in the form of extraneous material (paratext) that is interspersed throughout the content sentences that actually align across the bitext. Our 昀椀ndings point toward future research on better alignment algorithms which make fewer of these unrealistic assumptions for our task while remaining practical to execute.
All ancient Greek and Latin texts were extracted from The Perseus Proje2c3t][(Perseus) and contain no paratext. The translations varied in terms of language, sources, formats, available annotations, and o昀琀en included paratext as detailed in Tabl1e. The texts are further described by annotation level, since this impacted their use in di昀erent experiments. We aimed to test our pipeline on a varied set of texts, including di昀erent styles (dialogue, poetry, prose) and level of noisiness (texts with and without paratext). Therefore, we used available annotations, leading some experiments to be evaluated at the chunk level (Tab2l)e,sentence level (Table3), or at coarser levels (chapter or book: Tab4l).eThe one exception is the small test set (Tabl3e) that the authors of this paper manually annotated at the sentence level. We used this test set at the outset of our project for rapid testing in order to inform next steps. Details on the paratext present in the two noisy texts can be found in Tabl4e.
When we refer to “chunk-level,” we mean the most 昀椀ne-grained citation structure available on Perseus, such as chapters, sections, or Stephanus page4s. By “sentence-level,” we mean the phrases obtained a昀琀er sentence segmentation, explained below. Any additional pre- and post-processing is listed in the appendix.
Table2 lists all texts which were previously annotated at the chunk level. Preprocessing on
these texts was limited to concatenating the chunks into a continuous string. For the
sentencelevel experiments (Table3s, 4), we applied standard preprocessing to all texts: concatenated the
raw text into one string, then segmented into sentences. For languages supported by Stanza
[
4.2.1. Crito
Translator # Sentences
Avg. Tok./Sent. Std. Dev. Annotator
Students involved in a project at Lepizig University annotated thirteen translationsCorifto
(including two in English, one in German, and 昀椀ve in Persian), to match the chunk-level
annotation of the Greek text in Perseus. In our experiments, we used both English translations (by
Harold North Fowler15[] and Benjamin Jowett [
The two noisy translations in our dataset were annotated by The Open Greek and Latin Project. These texts are available to the public in XML 昀椀les that include varying levels of annotati6on. For the Thucydides French translation by Bétant26[], annotations include tags for paratext and book and chapter boundaries. For the Lucretius edition, which contains two English translations by Rev. John Selby Watson and John Mason Good13[], annotations include tags for paratext and book boundaries.
Both the Thucydides (fr) and Lucretius translations include paratext. In Thucydides, paratext consists of a foreword; commentary and summary of contents preceding each of the eight books; 33 footnotes; and an index. In the Lucretius edition, paratext consists of a foreword, a commentary before the prose translation, 1426 notes including 955 footnotes interspersed through the text (70 in the foreword, 861 in the Watson translation, 24 in the Good translation), and an index. The Lucretius edition contains additional noise in the form of two translations included in one edition.
Our experiments are summarized in Tabl5e. To guide the direction of our research, we 昀椀rst ran our two candidate pipelines, LASER-Vecalign and LaBSE-Vecalign, on tCherito test set. We then validated these results with retrieval experiments on our full dataset, using available annotations. Lastly, we tested our best pipeline, LaBSE-Vecalign, on noisy data with a focus 5The student translators used treebanks, commentaries, lexicon entries, and English and German translations to aid their Persian translations. They also aligned their Persian translations at the word level to the Greek, using Ugarit (Mohammadi’s can be found hereh:ttps://ugarit.ialigner.com/userProfile.php?userid=52434&tgid=9)3.62 6https://github.com/OpenGreekAndLatin
Scoring Functions (see 5.3) Recall Recall
Recall
Recall (coarse) Texts Used on error analysis to understand how the pipeline would fare on the type of unannotated data that we would like to align using our pipeline.
To run the retrieval experiments, we 昀椀rst passed pre-processed source and target texts segmented into chunks or sentences through an embedding model (LASER or LaBSE). Then we used cosine similarity to retrieve the most similar chunks or sentences across the bitext pairs. For chunk-level retrieval experiments, we le昀琀 out from the source text any chunks with missing translations (hence the di昀erent number of chunks across translations in Tab7le).
For LASER-Vecalign and LaBSE-Vecalign, we passed the chunk- or sentence-level embeddings into Vecalign, which outputs a set of predicted alignments. These may include one-toone, one-to-many, many-to-one, or many-to-many alignments.
For chunk-level retrieval, we report the percentage of source chunks with the correct result among the top 1 (correct result has the highest similarity score) and top 10 similarity scores. For sentence-level retrieval, we modify the numerator and denominator to account for oneto-many sentence alignments in the ground truth: for every Greek sentence, we compute the number of correct sentences retrieved divided by the number of target sentences in the true alignment.
For LASER-Vecalign and LaBSE-Vecalign, we evaluated results based on available annotations. For theCrito test set, where we manually produced sentence-level ground truth, we used Vecalign’s scoring functions and two new functions we formulated, described below. For chunk alignment experiments, where the ground truth is a straightforward list of one-to-one alignments, we report the percentage of incorrect predictions. Finally, for LaBSE-Vecalign on the two noisy texts, Thucydides (fr) and Lucretius (en), we evaluate the pipeline’s predictions at the coarse level of annotations available in the Open Greek and Latin Project’s database. These experiments were run on segmented sentences but we report accuracy relative to a predicted sentence belonging to the same chapter (Thucydides) or book (Lucretius) as the source text.
Vecalign’s original scoring function reports strict and lax scores for Precision, Recall, and F1. The lax metric expands the de昀椀nition of true positives to include any correct sentence alignment in a one-to-many or many-to-many prediction. A strict true positive requires exact matches between a ground-truth alignment and a prediction. When performing our initial rapid testing, we formulated two additional metrics. Both are based on post-processing Vecalign’s results by merging predicted alignments to try to reconstitute Perseus sections. We used this annotation level as ground truth because Perseus sections are examples of the available annotations applied to ancient texts by editorial convention. Therefore, we sought to determine how the pipelines would perform at this challenging level.
If a昀琀er merging there’s a strict match, then this is a true positive under the “New Strict Scoring” function. In the “New Lax Scoring” function, we instead look for lax matches: for any sentence that appears on both sides of the reconstituted alignment that is in a Perseus section, true positives are increased by one.
LaBSE outperformed LASER in our initial testing of the candidate pipelines, LASER-Vecalign and LaBSE-Vecalign. This was repeated in our validation experiments (chunk-alignment, chunk-retrieval, and sentence-retrieval).
When used in conjunction with Vecalign, LaBSE consistently outperformed LASER (Tab6l).e Unsurprisingly, both pipelines did best under the New Lax Scoring function. Since this metric gives credit for (counts as true positive) any correct sentence matching between source and target text, it most approximates the pipelines’ ability to correctly align sentences. Therefore, at the sentence-level, the results on clean data with thCerito were overall very promising. LaBSE’s and LASER’s worst results were with the Strict Scoring function, con昀椀rming our hypothesis that existing pipelines would struggle to return correct alignments at the more challenging annotation level that we 昀椀nd in ancient texts and their translations. Results for both pipelines at the chunk level are in Tabl1e2 in the appendix (LaBSE-Vecalign aligned all chunks correctly, while LASER-Vecalign aligned 98.88% of Greek - Fowler chunks and 94.98% of Greek - Jowett chunks correctly).
In the retrieval experiments, LASER struggled most when retrieving chunks from the English Thucydides translation, the longest text on which we tested LASER (Tab7l)e. LASER’s performance improved when retrieving sentences from thCerito test set (Table 8), in other words when retrieving shorter spans of text from a shorter document. We do not see similar di昀erence with LaBSE’s performance on shorter text spans and documents, with its scores at the sentence level slightly lower yet still comparable to those at the chunk level (T7a, ble Table8).
LaBSE and LASER both exhibited performance di昀erences across translations of thCerito in experiments using the test set and the full text. With the test set, LASER-Vecalign and LaBSEVecalign did best aligning the Greek to Fowler’s more literal translation (T6a),bwlehere there are the most one-to-one alignments in the ground truth (70% compared to 48% for Greek Jowett and Fowler - Jowett). Likewise, both LaBSE and LASER had higher scores retrieving sentences between the Greek and Fowler than between the Greek and Jowett. Interestingly, both models perform better retrieving sentences between Fowler and Jowett (“F-J” in Ta8b)le than when used in conjunction with Vecalign (“F-J” in Tabl6e). At the chunk level, LaBSE and LASER also did better on Fowler’s more literal translation in both chunk-level retrieval (Table7) and chunk-level alignment (Tabl1e2 in Appendix).
Testing LaBSE and LaBSE-Vecalign on noisy data points to the challenges of the two noisy features present in our noisy dataset: the presence of paratext and multiple translations in the target document. LaBSE did worse retrieving sentences from the noisy Thucydides translation (60.70% in Top 10 in Table9) than the English translation (93.15% in Top 10 in Tabl7e), even though the evaluation was done at a coarser level with the French. These experiments di昀er in three ways: the language of the target (French vs. English), the shorter text span in the sentence retrieval experiment, and the presence of paratext in the French. Given LaBSE’s comparable results with retrieval at the chunk and sentence level on tChreito, the presence of paratext is the likely reason for the performance di昀erence. At the coarser, book level we used to evaluate sentence retrieval with Lucretius, the presence of paratext no longer made a visible impact (Table9).
When used in conjunction with Vecalign, and also evaluated at the same coarse level, LaBSEVecalign did very well on Thucydides (fr), despite the presence of paratext. Thus 96.72% of paratext sentences correctly aligned to null and 94.04% of French text sentences were aligned to a Greek sentence from the same chapter (Table10). However, on Lucretius 35.03% of paratext sentences were incorrectly aligned to null and 55.23% of English text sentences incorrectly aligned to null (Tabl1e0). We also reported results using the number of Vecalign predictions as denominator, which can be found in Tabl1e3 in the Appendix and show a similar pattern.
In order to get results for Lucretius using LaBSE-Vecalign that are comparable to those on Thucydides, we had to suppress both paratext and the second translation (by Good) in the target edition. When we only suppressed paratext, 65.56% of Vecalign’s predictions were (incorrect) null-to-text alignments, and 55.21% of English text sentences were incorrectly aligned to null (Table 14 in the Appendix). When we only suppressed the second translation, 96.26% of text sentences were correctly aligned to Latin sentences from the same book, but 36.26% of paratext sentences were incorrectly aligned to Latin text sentences (Tab1l5e in the Appendix). Finally, when we suppressed paratext and only counted text sentences from the 昀椀rst translation (by Watson), 97.97% of English text sentences aligned to Latin sentences from the same book, and no English text sentences aligned to Latin sentences from a di昀erent book (Tabl1e1). In other words, only a昀琀er preprocessing the English edition of Lucretius to obtain a clean dataset (no paratext, no second translation) was LaBSE - Vecalign able to achieve results comparable to those we saw with the French translation of Thucydides.
Interestingly, when we compare against retrieval results in Tab9,leLaBSE did better on its own than it did with Vecalign on Lucretius, but worse on Thucydides. Thus using LaBSE embeddings only, for 97.24% of the 2428 Latin sentences we retrieved English text sentences from the same book among the top 10 most similar sentences, while LaBSE-Vecalign aligned only 44.24% of the 8521 English text sentences to Latin sentences from the same book. On the other hand with Thucydides, we retrieved French text sentences from the same chapter among the top 10 most similar sentences for 60.70% of the 6097 Greek sentences using LaBSE embeddings only, while LaBSE-Vecalign aligned 94.04% the 10958 French text sentences to Greek sentences from the same chapter. These results indicate that Vecalign’s faulty assumption and inductive biases for approximate dynamic programming can override the signal from well-trained sentence representation methods and hurt the overall performance in the case of discontiguous noisy text.
Our experiments led us to identify LaBSE-Vecalign as the pipeline using existing models best suited to produce sentence-level alignments of Ancient Greek and Latin texts with their translations. However, even with the clean data of thCerito, this pipeline evidenced di昀erences in performance across translations. This was even the case when aligning English to English (Fowler-Jowett), with LaBSE-Vecalign showing better results aligning Greek-Fowler than Fowler-Jowett. The performance driver wiCthrito seems therefore to be the nature of the translations themselves: the number of one-to-many and many-to-many alignments in the ground truth. The di昀erences in the two translations also emerge at the word level. We manually aligned the 昀椀rst half of the Crito’s words with Jowett’s translation on Ugarit31[] and compared to alignments with Fowler prepared by an author of Ugarit’s alignment guideli3n0e]s7.[ We were able to align 52% of Greek words with Jowett’s translation, compared to 80% with Fowler’s (Table16 in the Appendix). Of these aligned Greek words, with Jowett 9% crossed the predicted sentence boundaries while none did with Fowler’s (Ta1b7lein the Appendix).
With more realistic noisy data, LaBSE-Vecalign was not able to handle both the nature of the paratext in the Lucretius edition, and its inclusion of a second translation. The Lucretius edition not only has many footnotes, these are also lengthy. Thus the 昀椀rst two sentences of the translation in Lucretius (following 461 sentences of the foreword and 23 sentences of commentary preceding book 1) are followed by 44 sentences of footnotes before we 昀椀nd the third text sentence. In contrast, Thucydides had fewer footnotes (33), also interspersed in the translated text but all short (the longest spans 2 sentences).
Figures1 and 2 show details of the LaBSE-Vecalign results aligning Thucydides and Lucretius with their French and English translations, respectively. In the Thucydides detail, the Greek sentences on the le昀琀 are much longer than the French sentences; the 昀椀琀h Greek sentence (section 1.2.2) is translated over 昀椀ve sentences in French (last row of Figur1e). The 昀椀rst three rows are errors; the 昀椀rst two contain no overlapping sentences and the third includes the correct French sentences for section 1.1.3 as well as the French translations of about half of section 1.1.1 and all of section 1.1.2. There is no evident pattern explaining these errors. However, 7Word alignments with Jowetth:ttps://ugarit.ialigner.com/userProfile.php?userid=126388&tgid=12a0n65d Fowler: https://ugarit.ialigner.com/userProfile.php?userid=3&tgid=8609 with Lucretius, some errors can be explained (and excused), for example the 昀椀rst alignment in Figure2: the footnotes that are also aligned to the 昀椀rst sentence of the Latin include part of the translated text (“O Bountiful Venus”) and Latin original (“Alma Venus”). As with Thucydides, the remaining errors occurred across sentences covering similar subjects.
Figure2 suggests that the errors with Lucretius can be attributed to Vecalign: to correctly align all blue English sentences to the 昀椀rst Latin sentence, the algorithm would have to skip sentences 485-527 and then recognize the fragment in sentence 528 as part of the phrase begun in sentence 484. Vecalign was not designed to handle this case; its approximation to run in linear time averages the embeddings of consecutive sentences using a relatively small window (10 sentences in the default settings that we used). The blue sentences are too far apart in the English edition for Vecalign to capture them in one alignment. This is reminiscent of an error we encountered in the LASER-Vecalign results aligning Cthreito with Jowett’s English translation when a sentence in Jowett (in red) was out of order relative to the Greek (Fig3u).re
The extensive corpus of translations of the Greek and Latin classics holds tremendous promise for the study of translation, natural language processing, and variation and change in cultural assumptions. By studying both close translations and those full of literary license, paraphrase, censorship, and misunderstanding, we hope to enable scholars and students to understand this corpus better and translation-studies and NLP researchers to perform empirical studies of variation in translation. Our experiments demonstrate that a pipeline composed of state-ofthe-art NLP systems for performing automatic sentence alignment on literary text in ancient languages is useful but leaves a lot of room for improvement. Even our best-performing con椀昀guration struggles when translations exhibit signi昀椀cant discontiguity and non-monotonicity with respect to the source text. We expect this to be the case for the majority of extant translations that we would like to analyze computationally. The performance of our pipeline is further compromised by the presence of paratext—footnotes, commentaries, alternate translations, quotes from the source, and other extraneous material. This kind of noise in the form of paratext is also a common feature of collections of translations like the one compiled by the Open Greek and Latin Project that we aim to process, align at multiple granularities, and analyze computationally. Another example of noisy collections is the series of nineteenth-century Hachette editions of the classics, which alone contains 46 editions with more than one French translation each8.
Considering these factors, we perceive two directions for future work. The 昀椀rst is to improve Vecalign’s model to handle longer paratext and multiple translations by revisiting the assump8The Hachette series can be found on HathiTrust’s website: https://babel.hathitrust.org/cgi/mb?a=listis;c=1152044995 tions it makes in pruning its search space and redesigning the dynamic program for alignment. The second approach is to build classi昀椀cation systems 昀椀ne-tuned for the genre of classical translations. Since paratext annotations are not always available, this would allow us to automatically identify paratext and detect multiple translations before running books through our LaBSE-Vecalign pipeline. Fortunately, in the collection compiled by the Open Greek and Latin Project, some of this processing has been done, and the XML 昀椀les tag the paratext explicitly. It remains to build models to detect editions with multiple translations and facing source and target texts using page-level language identi昀椀cation to remove these violations of Vecalign’s continuity and monotonicity assumptions. We are currently preprocessing several translation collections of interest for running through our sentence alignment pipeline in order to release a large dataset linking the Greek and Latin classics and their translations at the sentence level.
We would like to thank the Open Greek and Latin Project for sharing their annotations of translations of Ancient Greek and Latin texts, as well as the student annotators of tChreito translations.
N. Reimers and I. Gurevych. “Making Monolingual Sentence Embeddings Multilingual Using Knowledge Distillation”. InP:roceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2020, pp. 4512–4525.
We installed LASER following instructions on the model’s GitHub p9agSee.ntence embeddings are the output of LASER’s encoder; we followed instructions for building them on the “embed” task page. Language-speci昀椀c encoders are not available for the languages in our dataset, therefore we defaulted to LASER2. We used default parameters.
We accessed LaBSE through its HuggingFace implementation under the SentenceTransformers clas1s0. We used default paramaters.
We installed Vecalign following instructions on its GitHub p1a1geR.unning Vecalign on sentence embeddings requires a few steps. First, build “overlap” 昀椀les from original text document (each row represents one sentence), or “concatenations of consecutive sentences” (each row represents one concatenation from one tno number of consecutive sentences). Second, embed the overlap 昀椀les (each row is an embedding of a concatenation). Third, align sentences using the embeddings of concatenated consecutive sentences.
We tested the impact of two parameters using theCrito test set for Ancient Greek - Fowler’s translation: number of overlaps (10 and 7) and maximum alignment size (8 and 5). Changing the parameters had no impact on the results, so we used Vecalign’s default values (number of overlaps = 10 and max alignment size = 8).
B.1. Crito
Both the Greek and the annotated translations were extracted at the chunk level, concatenated into a continuous series for use in chunk-level experiments, and segmented into sentences following our standard preprocessing for use in sentence-level experiments. The Greek we extracted from Perseus and the annotated translations from a google sheet shared with us. 9Installation instructions can be found on LASER’s GitHub homepage: https://github.com/facebookresearch/LASER/ 10Documentation and instructions for using LaBSE’s HuggingFace implementation can be found here: https://huggingface.co/sentence-transformers/LaBSE 11Installation and usage instructions can be found on Vecalign’s GitHub homepage: https://github.com/thompsonb/vecalign Before preprocessing, we removed newline characters that appeared to be errors in the text extracted from Perseus. A昀琀er segmentation, certain ending punctuation marks were split into the next sentence (e.g. right quote a昀琀er speech or square bracket in footnote). We corrected this by adding the punctuation mark back to the preceding sentence.
Before segmentation, we removed the whitespace that by convention appears between text and French quotation marksg(uillemets). Right guillemets were nevertheless frequently split onto their own sentence or appended to the beginning of the following sentence. We corrected this error a昀琀er segmentation.
Before preprocessing, we also removed newline characters that appeared to be errors in the text extracted from Perseus.
Text-to-text: at least one sentence from same chapter or book Errors: null-to-text or text-to-text from di昀erent chapter or book Thucydides Lucretius (chapter-level) (book-level) 12046
In Table 13, we report results of LaBSE-Vecalign on our noisy dataset using the number of predicted alignments as denominator.
Table14 reports results of LaBSE-Vecalign on Lucretius a昀琀er suppressing paratext only. We see continued errors in aligning English text sentences to null (55.21%).
In Table15, we report results a昀琀er suppressing only the second translation (by Good). In this experiment, we kept paratext in order to isolate the impact of multiple translations in the target text. These results show improved correct text-to-text alignments, with 96.26% of English text sentences aligned to Latin text sentences from the same book. However, the errors aligning English paratext sentences to Latin text sentences persist, with 36.26% of paratext sentences from the Watson translation aligned to Latin text sentences.
translations As a Percent of English Paratext Sentences From First Translation Only
Number of Paratext Sentences Paratext sentences to null (correct)
Paratext to Latin text (incorrect) As a Percent of English Text Sentences From First Translation Only
Number of Text Sentences
Text-to-text: to sources sentences from same book Text-to-text: to at least one source sentence from same book Errors, text-to-text: to no source sentence from same book
Errors, text-to-null
Lucretius (book-level) Percent of Aligned Greek Words that Cross Vecalign’s Predicted Sentence Boundaries, First 1,822 Words of Crito (through section 48a.4)