Fine-Tuning Pre-Trained Language Models for Authorship Attribution of the Pseudo-Dionysian Ars Rhetorica

Abstract

This paper explores the use of pre-trained language models for Ancient Greek in the context of authorship attribution. The study adopts a two-step approach: first, the models are fine-tuned on a domainspecific corpus using a masked language modeling (MLM) objective; second, based on the fine-tuned model, a classifier is trained to address the authorship attribution task. The analysis centers on a corpus of texts on rhetorical theory from the Second Sophistic period, with particular focus on the PseudoDionysianArs Rhetorica. The results of the experiment suggest that this approach ofers valuable insights into the authorship of ancient texts. Notably, the findings align with some traditional scholarly views on the Ars Rhetorica while also opening the door to reconsidering long-discarded hypotheses about the treatise's internal structure. This study highlights how the integration of natural language processing and classical philology can significantly advance discussions in ancient literary scholarship.

Keywords

• eol>pre-trained language models
• authorship attribution
• authorship analysis
• historical languages
• transfer learning
• ancient greek (roman period)
• Ps
• -Dionysius's Ars Rhetorica
• BERT
• RoBERTa

1. Introduction

Over the past several years, the application of transformer-based neural netwo5r1k]sh[as led to significant advancements in many NLP tasks related to historical languag4e4s,[ 32, 43 ]. However, unlike in the case of modern languages, where fine-tuning pre-trained transformers for linguistic forensics is very commo1n4,[ 48, 19, 1 ], the application of such models for authorship attribution tasks in historical languages remains relatively underexplored, although some excellent seminal studies and surveys have been recently publish4e5d,1[ 5, 40, 41 ]. The availability of state-of-the-art pre-trained language mode2l,s32[ , 39, 54 ] excelling in multiple downstream tasks suggests that the situation with authorship analysis can be diferent as well.

Yamshchikov, Tikhonov, Pantis, Schubert, and Jost54[] obtained a pre-trained model for Ancient Greek by fine-tuning a Modern GreekBERT model [ 23 ]. The resulting model subsequently served as the backbone for a classifier and proved efective for authorship attribution of the so-called Pseudo-Plutarch corpus. Interestingly, despite being fine-tuned on a limited amount of Ancient Greek data, the model obtained through transfer learning showed results comparable to those of the models trained from scratch on significantly larger corpora, as reported by Singh, Rutten, and Lefever 3[ 9 ] and Riemenschneider and Frank32[]. Drawing inspiration from Yamshchikov, Tikhonov, Pantis, Schubert, and Jos5t4[], this study experiments with a similar approach focusing on the works of late Greek rhetoricians.

Greek prose on rhetorical theory from the period known as the Second Sophistic serves as a crucial source, documenting the cultural and intellectual framework of Greek thought and literature in the first centuries AD9[ , 6, 20, 5 ]. However, the study of this extensive corpus of texts, collectively referred to under the broad concepRthoeftores Graeci [ 52, 42 ], is significantly complicated by endless controversies surrounding authorship, dating, and contextual factors [ 21 ].

In this paper, we explore the potential of a transformer-based models, fine-tuned for sequence classification task, to provide further insights into the debate.

The focal point of our study is the text conventionally referred to asAtrhseRhetorica (Art of Rhetoric, hereafter ars). This work has long been attributed to, and frequently published under the name of, the rhetorician and historian Dionysius of Halicarnascsau.s6(0–7 BC). However, Sadée [ 36 ], followed by Usener4[ 9 ] and Usener and Radermacher5[0], demonstrated that the ars most likely circulated anonymously, with its association to Dionysius emerging from a much later conjecture. This conjecture appears to have been based on an overinterpretation of a scholion (a marginal commentary) on chapt1e0r of the text.

2. Ars Rhetorica

Several aspects of thears must be discussed in the context of statistical modelling of its writing style.

2.1. Not one, but multiple works

The text has a complex structure. InParisinus Graecus 1741 [ 30 ], the only manuscript that preserves all the material associated with thaers (f. 1–37), the text is divided into 11 chapters. However, these chapters do not constitute a homogeneous work, as the text is generally understood to consist of two18[], three [ 49, 50, 35, 33 ], or even four 3[ 8 ] distinct parts.

The first part, covering ch. 1–7, provides concise instruction on ceremonial (epideictic) oratory, addressing seven epideictic genres. These chapters are connected by cross-references and recurring addresses to the author’s former pupil, Echecrates, to whom the text is presented as a wedding gift. The remainder of the text, ch.8–11, may be interpreted as a combination of two or three distinct works on separate topics. C8h–.9 explore the so-called “figured speeches”, i.e., speeches intended to convey a hidden meaning that may conflict with the literal content and stated purpose of the speech, while ch1.0–11 focus on the criticism of declamation.

2.2. Authorship

Ch. 1–7 exhibit a consistent compositional pattern and a recognizable writing style, suggesting they were authored by the same rhetorician. However, whether these chapters form a coherent and complete treatise is a matter of debate. This portion of tahres has been interpreted as a collection of distinct letters or essay38s,[ 18 ], as remnants or excerpts from a much longer work [ 4, 49, 50 ], or as a unified treatise [ 53, 47, 22 ]. For ch.8–11, the situation is even more ambiguous. Usener [ 49 ] speculated that ch.8–9 were written by two diferent disciples attending separate lectures of the same teacher. Penndor2f8[] and Schöpsdau [ 37 ] rejected the idea that ch.8–9 had a single author, suggesting instead that these texts drew from various sources. Similarly, ch. 10–11 have been attributed either to the same author as ch8–.9 (with Heath [ 18 ] tentatively identifying him as Sarapion Aelius, a 2nd-century Alexandrian rhetorician whose entire corpus is lost) or to two diferent authors unrelated to the rest of tahres. Table 1 summarizes the content and authorship hypotheses for the various sections of tahres.

2.3. Ars Rhetorica, Menanderian Corpus, and Pseudo-Hermogenes’ On Method

Since the early days of scholarship on thAers Rhetorica, it has been noted that the rhetorical instruction provided in ch1–.7 and ch.8–11 shows a clear methodological afÏnity with, respectively, the treatises ascribed to Menander Rhetor (particularly the second one) and PseudoHermogenes’On Method. The parallels with the second treatise attributed to Menander are especially noteworthy. In both works: • the occasion — rather than the subject, as in the first treatise attributed to Menander — determines the genre; • a very similar selection of genres is discussed (of the seven genres mentioned inartsh, e only two are absent from Menander’s purported work; see Ta7b);le • the author addresses a former disciple throughout the text. This afÏnity led Heath [ 18 ] to describe thears as “comparable to, though less sophisticated than” Menander’s work.

The numerous parallels between c8h–.11 of the ars and Pseudo-Hermogenes’On Method [ 29, 18 ] have led scholars to hypothesize either a shared sour2ce9][or a closer, albeit indirect, connection [ 35, 18 ]. 2.4. Dates The dates of the texts constituting thears have been assessed diferently. For ch.1–7, a mention of the 2nd-century sophist Nicostratus (ch2., par. 9, p. 266, l. 14), along with the considerable focus on speeches addressing Roman magistrates, suggests a composition date no earlier than the High Empire [ 35, 22 ]. Race [ 31 ] posits that the first part of the ars is roughly contemporary with the corpus attributed to Menander Rhetor, which is datable to the late 3rd century AD. In contrast, ch.8–11 may be a century earlier1[ 8 ], i.e., 2nd century AD.

3. Aims

The hypotheses concerning the authorship of diferent parts of thaers have multiplied, as have suggestions regarding its potential relationship with other texts. However, the evidence presented in the scholarship so far is drawn almost exclusively from close reading and remains inconclusive. Additionally, unlike the case with8c–h1.1, no eforts have been made to identify the author responsible for ch1–.7.

The aim of our investigation, therefore, is to apply modern natural language processing techniques to this rich textual material in order to gather new evidence about the structure of the ars and gain further insights into its authorship. The arguments formulated through language modeling could provide a novel and valuable contribution to the debate, particularly when considered alongside the accumulated philological evidence and existing codicological indications.

The main contributions of this work can be summarized as follows: • We further fine-tune two pre-trained models for Ancient Greek and one model for Modern Greek on a corpus of Greek rhetoricians. We subsequently use the resulting models to train “open set”10-class classifiers capable of attributing short fragments of text to diferent authors of the Second Sophistic period; • Analyzing in more details the results provided by two best-performing models, we shed light on the history of the Pseudo-Dionysiaanrs, suggesting that: – Ch. 1–7 of the ars could have been authored by an individual from the same school as the author(s) of the Menandrian treatises; – Ch. 8–11 not only difer in authorship from ch1.–7, but may have been written by two distinct individuals, one responsible for8c–h9. and another for ch1.0–11.

4. Corpus

The primary focus of our study is a corpus comprising at le1a8strhetores Graeci from the 1st–4th centuries AD and Dionysius of Halicarnassus. We only retained the authors whose teachings are relatively well-preserved, excluding those known only through fragmentary or indirect evidence. Importantly, we focus exclusively on rhetorical theory, i.e., works with a theoretical or pedagogical intent.

A significant limitation of this dataset is that many of the rhetorical corpora within it have notorious attribution problems of their own. In particular, there is a compelling case for the heterogeneity of the Hermogenean corpus (see Sectio6n). Similarly, the question of whether both treatises attributed to Menander were authored by the same person remains unresolved [ 34, 17, 31, 8 ]. Other corpora raise similar questions, to2o1][. Being aware of this and currently working on a follow-up authorship verification study of these corpora (the importance of which was also insightfully emphasized by the reviewers of this work), for simplicity, we continue to group the studied texts by authorship as categorized in Tthheesaurus Linguae Graecae (TLG), where our dataset stems from1 . We deem this simplification legitimate. In most cases, these questionable attributions are rooted in long-standing traditions that date back to the early stages of textual transmission. For example, the Hermogenean corpus has been consistently attributed to Hermogenes of Tarsus since as early as the 5th century (for more details see Section6). Therefore, with all necessary reservations, these conventional groupings can be considered to represent at least some kind of connection. Even if they do not link texts written by the same individual, they may still group works originating from the same school. After all, this is why such simplification is commonly used in scholarship.

4.1. <UNK> category

The literature on oratory theory was undoubtedly much richer than what has been preserved. To account for this, we created an “open set” scenario. For this purpose, we set as9idsmealler authorial corpora — those with a number of sentences below the dataset’s median valu5e17of (marked with * in Table2). These texts were excluded from the dataset before our conventional 80/10/10 split and later added to the the test set. The idea is straightforward. If at the test stage the model encounters a text that does not belong to any of the authorial classes learned during the training, it is likely that the calibrated probability associated with the top prediction will be relatively low. If it falls below a certain threshold, the model is programmed to abstain from making a decision and assign an<UNK> label to the text in question. The samples wit<hUNK> label in the test split are necessary to monitor the model’s capability to do.

An overview of the classification dataset is presented in the Tabl2e. 1We cannot publish the full texts with all the corresponding metadata. However, the shufÒed chunks used in MLM ifne-tuning and subsequent classifier training are made available on GitHubh:ttps://github.com/glsch/rhetores_ graeci.

5. Methodology

5.1. Base Transformers

To train our classifiers, we used three diferent pre-trained transformers as starting points: (1) RoBERTa-sized GreBerta presented by Riemenschneider and Fran3k2][, (2) Ancient GreekBERT trained by Singh, Rutten, and Lefever3[ 9 ] (3) Modern GreekBERT published by Koutsikakis, Chalkidis, Malakasiotis, and Androutsopoulos [ 23 ].

5.2. Masked Language Modeling Fine-Tuning

Fine-tuning pre-trained models on domain-specific corpora prior to further tuning them for a downstream task at hand is a common practice in NLP. It allows the model to adapt better to the unique linguistic features of the target domain. This intermediate step may enhance the model’s ability to capture specific syntactical patterns and vocabulary, which in turn improves the performance on the final downstream task, such as classification. For this reason, before training classifiers for authorship attribution, we ran training with a masked language modeling objective. BERT-sized models were trained fo3repochs with a learning rate1 × 10−5 and warmup during the first 10% of training steps.RoBERTa-based model was trained fo1r epoch only with a learning rat1e× 10−4 and without warmup steps. In both scenarios, the learning rate was decreasing linearly.

5.3. Sequence Classification

Authorship classifiers were trained on both out-of-the-box models and their MLM-fine-tuned versions. We employed a sliding window technique to segment the texts into chunks. The process was as follows: 1. Tokenization: We used thebowphs/GreBerta tokenizer to convert the entire corpus into tokens.2 2. Chunking: The tokenized corpus was then divided into chunks6o4ftokens, respecting the boundaries of works (and even chapters — in the case of tahres); 3. Overlap: To ensure continuity and capture context that might span chunk boundaries, we implemented an overlap between chunks. Each chunk overlapped with its adjacent chunks by 32 tokens (half of the chunk length). 4. Decoding: Finally, we decoded these token chunks back into text, resulting in our training data segments. By using a single tokenizer to chunk the entire corpus beforehand instead of splitting the texts with a tokenizer of the corresponding model, we ensured that all models were trained on the same segments of text.

The training was carried out fo7r00 steps by sampling batches containing4 chunks per authorial class. Validation set was checked eac3h50 steps, i.e., twice during the training. Test set including<UNK>-labelled samples was checked upon the end of training. We report the results obtained on the test set.

6. Results and Discussion

6.1. General Performance

Table3 summarizes the overall performance of the classifiers. Notably, additional MLM training proved beneficial only for theRoBERTa-sized bowphs/GreBerta model. ForBERT-sized models, however, the inclusion of new data was detrimentbaolw.phs/GreBerta appears to be more stable, behaving more like general-purpose language models trained for well-resourced modern languages. This stands to reason: out of the three models with which we experimented, bowphs/GreBerta [ 32 ] is the largest and was trained on the riches and highest-quality Ancient Greek corpus.

6.2. Authorship Attribution of the ars

The aim of this study was to get some fresh evidence about the authorship of the pseudoDionysianars, a precious witness to the development of rhetorical theory during the High to Late Roman empire. Based on thestatus quaestionis surveyed in the section2, we set up 3 research questions: 2We did not repeat the experiment producing chunks with other available tokenizers. 1. Can we further comfort or challenge the existing consensus opinion, according to which the attribution to Dionysius of Halicarnassus is incorrect? 2. How many works are discernible in thaers in the form we know it? 3. Can the model convincingly suggest an alternative attribution foratrhseor any of its parts?

To address these questions, we applied the trained classifier to individual chapters of the ars, split into chunks following the described procedure. Ta4bsluemmarizes the predictions made by the best-performingBERT-sized andRoBERTa-sized models3. For each chapter, we report the “majority vote” (i.e., the number of chunks in the chapter attributed to a given author), the author’s “share” (i.e., the proportion of chunks assigned to that author in the total number of chapter chunks), and the mean probability of the author across the chunks of the chapter. In the “majority vote”, the attribution is defined by the top probability even if it falls bel8o0w%.

6.3. No trace of Dionysius of Halicarnassus

In line with previous scholarship, although the name of Dionysius of Halicarnassus appears among the attributions, its weight is insignificant. Therefore, with regard to the first of the research questions, the evidence is overwhelming: stylistic afÏnity with Dionysius of Halicarnassus’s writings is scarce, and the attribution to him cannot be supported by any of the two models.

6.4. ars’s association to the Menandrean corpus further strengthened

Apart from this rather predictable conclusion, our classifiers yield new insights into more complicated questions concerning the inner structure of athrse and the authorship of the texts, which constitute it. As clear from the Tabl4e, the attribution profiles for ch.1–7 and 8–11 are drastically diferent. Even when the probability is not high enough, Menander Rhetor is the top-ranked candidate in ch1.–7. The signal is less clear in ch8.–11. This diference goes in 3pranaydeeps/Ancient-Greek-BERT and bowphs/GreBerta (R)

Menander Aelius Aristides Dionysius H.

Rest Menander Dionysius H.

Aelius Aristides Rest Menander Dionysius H.

Hermogenes Rest Menander Hermogenes Aelius Aristides Rest Menander Aelius Aristides Dionysius H.

Rest Menander Hermogenes Dionysius H.

Rest Menander Aelius Aristides Dionysius H.

Rest Hermogenes Valerius Apsines Aelius Aristides Rest Hermogenes Demetrius Aelius Aristides Rest Hermogenes Dionysius H.

Valerius Apsines Rest Hermogenes Menander Dionysius H.

Rest “Majority vote”, share, and mean prediction probability for each chapter of the ars: Ancient Greek BERT (R). “Rest” stands for the sum of all minor attributions. Sorted by the mean prediction pranaydeeps/Ancient-Greek-BERT

bowphs/GreBerta (R) of ch. 1–7 being yet another argument in favour of its unity. line with thecommunis opinio that the work is composite: a nearly identical attribution profile

6.5. What does the model learn?

For the sake of explainability, DH specialists still widely use the bag-of-words model and corpus-specific manual feature engineering for various tasks involving writing style analysis, such as authorship attribution, authorship and self-authorship verification, clustering, e1t3c,. [ 12, 24, 3 ]. Since deep learning methods lack this level of transparency, understanding exactly

Menander Aelius Aristides Dionysius H.

Rest Menander Aelius Aristides Dionysius H.

Rest Menander Hermogenes Dionysius H.

Rest Menander Aelius Aristides Demetrius Rest Menander Aelius Aristides Hermogenes Rest Menander Valerius Apsines Dionysius H.

Rest Menander Aelius Aristides Valerius Apsines Rest Hermogenes Aelius Aristides Dionysius H.

Rest Aelius Aristides Hermogenes Dionysius H.

Rest Hermogenes Dionysius H.

Valerius Apsines Rest Hermogenes Menander Dionysius H.

Rest what our classifier learned is crucial. A thorough investigation of this matter will be the subject of a separate study, using explainable AI techniques such as integrated gradients and token attribution. Here, we limit our discussion to one insightful example, which seems to illustrate how the model works.

As previously mentioned in Sectio2n, all the genres addressed in c2h–.5 are also discussed in the second treatise attributed to Menander Rhetor. Only the most prestigious of the epideictic genres, the panegyric — focused on in ch1. and 7 — does not correspond to any section in Menander’s works. However, ch1., which provides introductory notes on panegyrics, often echoes the examples and some wording of the first treatise by Menander. C1h.ofers guidelines on how to appropriately praise gods (“leaders and name-givers of any festival”), cities where the festivals take place, and emperors who organize and preside over the festivals. All these topics are covered in Menander’s first treatise.

Considering only ch2.–5 or the fragments of ch.1 that have clear parallels in Menander’s work, one might argue that the classifier’s decision was biased due to the significant content and semantic overlap, especially since such a tendency has been reported aboutBtEhReT-based classifiers [ 7 ]. However, the consistency of the attribution profile across the chapters by both models is reassuring, as it suggests that they capture more than just semantics.

Menandrean association appears all the stronger when the values for the logical subdivisions of the ars, ch. 1–7, are calculated. As Korenjak22[] has shown, in its current form, the order of the chapters is disorganized, and it is possible that the author intended to arrange them as follows: chapters1 and 7 (panegyrics or appraisal speeches), chapter2s–4 (speeches related to family life occasions), and chapter5s–6 (speeches addressed to ofÏcials and epitaphs). In each of these sections, Menander maintains a stable leadership (Tab5leasnd 6). ars, ch. 8–11: multiple authors? The discrepancy between the attribution profiles of ch8.– 9 and ch.10–11 might suggest a division, albeit a less distinct one, than c1h–.7 versus ch.8–11. This result aligns with the assessment made by Usene4r9[], although it does not provide any further hint at the identity of the possible author.

However, the opposite hypothesis should still be considered seriously. In1–c7h,.top two single attributions (i.e., Menander Rhetor and another author) in terms of “share” would cover at least 0.58–0.68 of the attributed chunks (ch7.). In contrast, the top two attributions in ch8–. 11 provide,at best, 0.58–0.62 of the attributions (ch1.1 and10), the attributions are more evenly distributed. Apparently, among the author classes present in our dataset, none is stylistically similar enough to the text of ch8–.11. This can be explained in two ways. Texts written in a comparable style are either completely absent from the dataset or are not appropriately distributed among author groups, making it challenging for the model to learn the features of this particular writing style. Keeping in mind the existing hypothesis about the relationship between the so-called Hermogenean canon and the works ascribed to Apsines, with extreme caution, we incline to the latter explanation.

Two works, which are part of the Hermogenean canoOn,n Invention and On Method, were already in Late Antiquity associated with the name of Hermogenes. In our dataset, therefore, following the TLG, we reproduce this conventional attribution. Yet, both are most likely inauthentic [ 10, 11, 25, 26, 27 ]. If the argumentation presented by Heath1[ 6, 17 ] proves correct and these two texts can securely be ascribed to Apsines, the “new” writing style they would represent might possibly demonstrate a more pronounced afÏnity with the style of c8h–.11. This and similar possibilities should be thoroughly checked in further experiments.

The scope of the much-needed detailed follow-up study becomes evident. A systematic and critical reassessment of attribution problems within the corpus ofRthheetores Graeci is necessary. Beyond merely reflecting on the attributions of individual works, it is important to establish the homogeneity of diferent rhetorical corpora within the framework of a pairwise authorship verification study.

But if we set aside the obscure case of ch8.–11, should we conclude that ch1.–7 were written by Menander Rhetor? Given the aforementioned limitations of our dataset, we would not go that far. However, our results suggest that the connection between the first part of the Pseudo-Dionysianars and the Menandrean corpus likely extends beyond a theoretical afÏnity. Despite the obvious terminological discrepancies between the texts and their diferent levels of elaboration, the possibility of multiple authorship within the same school, or even common authorship, should be considered with all seriousness. The divergence betweenartsh,ceh. 1–7, and the Menandrean corpus can also be explained, apart from the natural evolution of personal style and preferences, by the likelihood that those presenting complex rhetorical theory would probably follow the advice formulated by the author of 1c1h.. The art of rhetoric involves presenting material in a way that convinces the audience. Thus, orators are similar to doctors who must not only select the right medication but also administer it in a manner acceptable to the patient [50, ch. 11, par. 9, p. 385, ll.7–12]. In other words, multiple contextual factors influenced the style of the presentation, and, in the cases when the stylistic afÏnity is clear, one should not probably overinterpret isolated diferences.

7. Conclusion

This study uses transformer-based models to analyze ancient rhetorical texts for authorship attribution in classical philology. First, we adapted these models to handle the linguistic nuances of Ancient Greek texts from the 1st to the 4th century AD using masked language modeling. We then apply the fine-tuned models to identify authorship markers iAnrs Rhetorica, a text possibly written by multiple ancient writers. This application not only reminds of benefits of modern AI techniques to classical studies but also deepens our understanding of ancient literary compositions through modern computational methods.

The results of BERT and RoBERTa classifiers do not support connection of tahres to Dionysius of Halicarnassus, going in line with the previous studies that question his authorship. They also strengthen the link oafrs to the Menandrean corpus, particularly evident in the distinct attribution profiles between chapters1–7 and 8–11, which suggests a composite nature of the work.

Despite the lack of transparency of MLM techniques compared to conventional methods, which prioritize human-interpretable features, the efectiveness and relevance of machine learning methods is noteworthy.

While neural networks are often criticized in digital humanities for their black-box nature [ 12 ], their ability to detect writing styles make them a valuable tool in the field of digital humanities. The use of these models promises significant advancements in authorship attribution and our understanding of ancient literary works.

8. Limitations

This study has several limitations that should be considered when interpreting the results.

Firstly, the issue of disputed authorship within the dataset is a significant challenge. For instance, the Hermogenean corpus and Menandrean treatises, both central to our analysis, have long-standing debates regarding their true authorship, see Sect4io.nThese uncertainties could afect the attribution accuracy. We are currently working on a study intended to solve this issue, adopting an authorship verification approach.

Secondly, the use of transformer-based models lBikEReT andRoBERTa, come with limitations related to their opaque nature. The lack of interpretability in these models means that understanding the specific features and patterns the models use to make attributions is challenging. This limits our ability to provide a transparent rationale for the models’ decisions, which is often critical in digital humanities research. Yet, the attempts were made to find way to make the results of pre-trained language models more interpretable, e.g., by means of the so-called integrated gradients4[ 6 ]. These methods can perhaps be adapted for cases similar to ours.

Despite achieving notable accuracies with relatively short chun6k4s t(okens), the models’ performance still leaves room for improvement, particularly in terms of handling unbalanced corpora and downplaying the influence of the thematic clues. Nevertheless, their performance, comparable to state-of-the-art results for modern languages, demonstrates an ability to capture writing style. There clearly are instances where the models are overly confident, leading to incorrect authorship attribution. These errors could arise from factors such as the models’ sensitivity to stylistic nuances and the complexity of the texts. Embracing more sophisticated methodologies for uncertainty-aware training would be an interesting avenue for further exploration.

Another potential avenue for future research is the development of chronological and regional classifiers. Texts from diferent regions and periods may exhibit unique linguistic and stylistic features that are not captured by a generalized model. Developing classifiers specific to historical periods or geographical (and cultural) areas could enhance attribution accuracy and ofer more detailed insights into thears and many other texts.

Acknowledgments

We extend their gratitude to Jürgen Jost, Charlotte Schubert, Friedrich Meissner, Caroline Macé, and Mark de Kreij for welcoming this study and future collaboration between machine learning, history, and philology.

We would also like to thank Ben Nagy and two anonymous reviewers for the careful reading and insightful feedback.

We thank Shari Boodts and Sven Meeder, Principal Investigators of the ERC Proof of Concept project “ManuscriptAI” and the ERC Consolidator project “SOLEMNE”. Without their support, this research would not have been possible. [33] D. A. Russell. “Rhetors at the Wedding”. en. InPr:oceedings of the Cambridge Philological Society 25 (1979), pp. 104–117. issn: 0068-6735, 2053-5899. doi: 10.1017/S0068673500004 156. (Visited on 05/07/2024). The code and both models considered in detail in this study are accessible at: • https://huggingface.co/glsch • https://github.com/glsch/rhetores_graeci

References

[1] B. Ai , Y. Wang , Y. Tan , and S. Tan . Whodunit? Learning to Contrast for Authorship Attribution . 2022 . doi: 10 .48550/ARXIV.2209. 11887 . (Visited on 01/11/ 2024 ).

[2] D. Bamman and P. J. BurnsL.atin BERT : A Contextual Language Model for Classical Philology . 2020 . url: https://arxiv.org/abs/ 2009 .10053.

[3] P. Beullens , W. Haverals , and B. Nagy . “ The Elementary Particles: A Computational Stylometric Inquiry into the Mediaeval Greek-Latin Aristotle” .MIned:iterranea. International Journal on the Transfer of Knowledge 9 ( Apr . 2024 ), pp. 385 - 408 . issn: 2445 - 2378 . doi: 10 .21071/mijtk.v9i. 16723 . (Visited on 05/06/ 2024 ).

[4] F. Blass. De Dionysii Halicarnassensis scriptis rhetoricis . Bonn: Max Cohen et fil., 1863 . url: https://books.google.nl/books?id=k3g- AAAAcAA .J

[5] B. E. Borg . Paideia: the World of the Second Sophistic . de Gruyter, 2008 .

[6] G. W. Bowersock . Greek sophists in the Roman Empire . eng. Oxford: Clarendon Press, 1969 . isbn: 978 -0- 19 -814279-9.

[7] F. Brad , A. Manolache , E. Burceanu , A. Barbalau , R. Ionescu , and M. PopesRcuet.hinking the Authorship Verification Experimental Setups . 2022 . url: https://arxiv.org/abs/2112.05 125.

[8] K. Brodersen .Menandros. Abhandlungen zur Rhetorik. ger grc . Vol. 88 . Bibliothek der griechischen Literatur. Stuttgart: Anton Hiersemann , 2019 . isbn: 978 -3- 7772 -1934-9.

[9] T. C. Burgess . Epideictic literature . Vol. 3 . University of Michigan Library, 1902 .

[10] E. Bürgi. “ Ist die dem Hermogenes zugeschriebene Schrift Περὶ μεθόδου δεινότητος echt? I.” In: Wiener Studien 48 ( 1930 ), pp. 187 - 197 .

[11] E. Bürgi. “ Ist die dem Hermogenes zugeschriebene Schrift Περὶ μεθόδου δεινότητος echt? II.” In: Wiener Studien 49 ( 1931 ), pp. 40 - 69 .

[12] T. Clérice and A. Glaise . “ Twenty-One * Pseudo-Chrysostoms and more: Authorship Verification in the Patristic World” . In:Computational Humanities Research Conference 2023. Proceedings of the Computational Humanities Research Conference 2022. Dec . 2023 . url: https://inria.hal. science/hal-0421117 . 6

[13] S. Corbara , A. Moreo , F. Sebastiani , and M. TavonMi.edLatinEpi and MedLatinLit: Two Datasets for the Computational Authorship Analysis of Medieval Latin Texts . Sept. 2021 . url: http://arxiv.org/abs/ 2006 .12289(visited on 02/05/ 2024 ).

[14] M. Fabien , E. Villatoro-Tello , P. Motlicek , and S. Parida . “ BertAA : BERT Fine-Tuning for Authorship Attribution” . InP:roceedings of the 17th International Conference on Natural Language Processing (ICON) . Ed. by P. Bhattacharyya , D. M. Sharma , and R. Sangal . Indian Institute of Technology Patna, Patna, India: NLP Association of India (NLPAI), Dec . 2020 , pp. 127 - 137 . url: https://aclanthology.org/ 2020 .icon-main.. 16

[15] B. Graziosi , J. Haubold , C. Cowen-Breen , and C. Brooks . “ Machine Learning and the Future of Philology: A Case Study” . en. InT:APA 153 .1 ( Mar . 2023 ), pp. 253 - 284 . issn: 2575 - 7199 . doi: 10 .1353/apa. 2023 . a901022 . (Visited on 10/20/ 2024 ).

[16] M. Heath . “Hermogenes' Biographers”. InE:ranos 96 ( 1998 ), pp. 44 - 54 .

[17] M. Heath . Menander: a Rhetor in Context . Oxford University Press, USA, 2004 .

[18] M. Heath . “ Pseudo-Dionysius Art of Rhetoric 8-11: Figured speech, Declamation, and Criticism” . In:American Journal of Philology 124.1 ( 2003 ), pp. 81 - 105 .

[19] J. Huertas-Tato , A. Huertas-Garcia , A. Martin , and D. CamachoPA .RT: Pre-Trained Authorship Representation Transformer. Sept . 2022 . url: http://arxiv.org/abs/2209.15373 (visited on 01/11/ 2024 ).

[20] G. A. Kennedy . Greek Rhetoric under Christian Emperors . Vol. 3 . Wipf and Stock Publishers, 2008 .

[21] G. A. Kennedy. “ Some Recent Controversies in the Study of Later Greek Rhetoric” . In: American Journal of Philology 124.2 ( 2003 ), pp. 295 - 301 .

[22] M. Korenjak . “ Ps .-DionysiusArs RhetoricaI-VII : One Complete Treatise” . In:Harvard Studies in Classical Philology 105 ( 2010 ), pp. 239 - 254 .

[23] J. Koutsikakis , I. Chalkidis , P. Malakasiotis , and I. Androutsopoulos. “ Greek-BERT: The Greeks Visiting Sesame Street” . In:11th Hellenic Conference on Artificial Intelligence . SETN 2020 . Athens, Greece: Association for Computing Machinery, 2020 , pp. 110 - 117 . isbn: 9781450388788 . doi: 10 .1145/3411408.3411440.

[24] N. Manousakis and E. Stamatatos. “ Authorship Analysis and the Ending of Seven Against Thebes: Aeschylus' Antigone or Updating Adaptation?” en . ICnl:assical World 116.3 ( Mar . 2023 ), pp. 247 - 274 . issn: 1558 - 9234 . doi: 10 .1353/clw. 2023 . 0007 . (Visited on 02/01/ 2024 ).

[25] M. Patillon . “ LeDe Inventione du Pseudo-Hermogène” . InT:eilband Sprache und Literatur . Einzelne Autoren seit der hadrianischen Zeit und Allgemeines zur Literatur des 2. und 3 . Jahrhunderts . Vol. 34 /3. Aufstieg und Niedergang der römischen Welt. Berlin, Boston: De Gruyter, 1997 , pp. 2064 - 2172 . isbn: 9783110815146 . doi: 10 .1515/ 9783110815146 - 003 .

[26] M. PatillonP. seudo-Hermogène, L 'Invention. Anonyme, Synopse des exordes . grc fre . Vol. 3 , 1 . Corpus rhetoricum. Paris: Les Belles lettres, 2012 .

[27] M. PatillonP. seudo-Hermogène, La méthode de l'habilité. Maxime, Lex objections irréfutables . Anonyme, Méthodes des discours d'adresse. grc fre . Vol. 5 . Corpus rhetoricum. Paris: Les Belles lettres, 2014 . isbn: 978 -2- 251 -00591-1.

[28] J. Penndorf . “ De sermone figurato quaestio rhetorica” . InL:eipziger Studien zur classischen Philologie 20 ( 1902 ), pp. 169 - 194 .

[29] H. Rabe . Hermogenis Opera. Teubner, 1985 . isbn: 978 -3- 519 -01760-8. url:https://books .google.nl/books?id=WreAtwEACA A.J

[30] H. Rabe . “ Rhetoren-Corpora” . InR:heinisches Museum 67 ( 1912 ), pp. 321 - 357 .

[31] W. H. Race . Menander Rhetor. Dionysius of Halicarnassus, Ars Rhetorica. grc eng . Vol. 539 . Loeb classical library . Cambridge (Mass.) London: Harvard University Press, 2019 . isbn: 978 -0- 674 -99722-6.

[32] F. Riemenschneider and A. Frank . “ Exploring Large Language Models for Classical Philology” . en. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . Toronto, Canada: Association for Computational Linguistics, 2023 , pp. 15181 - 15199 . doi: 10 . 18653 / v1 / 2023 . acl - long . 846 . (Visited on 02/10/ 2024 ).

[34] D. A. Russell and N. G. WilsonM. enander Rhetor. grc eng . Oxford: Clarendon Press, 1981 . isbn: 978 -0- 19 -814013-9.

[35] D. A. Russell . “ Classicizing Rhetoric and Criticism: the Pseudo-Dionysian Exetasis and Mistakes in Declamation” . InL:e Classicisme à Rome aux 1ers siècles avant et après J .-C 25 ( 1979 ).

[36] L. Sadée. De Dionysii Halicarnassensis scriptis rhetoricis quaestiones criticae . lat. Strasbourg: Teubner , 1878 . isbn: 978 -0- 666 -72899-9.

[37] K. Schöpsdau . “ Untersuchungen zur Anlage und Entstehung der beiden pseudodionysianischen Traktate περὶ ἐσχηματισμένων” . In:Rheinisches Museum für Philologie 118.H. 1/2 ( 1975 ), pp. 83 - 123 .

[38] H. Schott. ΤΕΧΝΗ ΡΗΤΟΡΙΚΗ : quae vulgo integra Dionysio Halicarnassensi tribuitur, emendata, nova versione Latina et commentario illustrata . Sumtibus E.B. Suicquerti , 1804 . url: https://books.google.nl/books?id=SiYUAAAAYAA.J

[39] P. Singh , G. Rutten , and E. Lefever. “ A Pilot Study for BERT Language Modelling and Morphological Analysis for Ancient and Medieval Greek”P. rIonc:eedings of the 5th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage , Social Sciences, Humanities and Literature. Ed. by S. Degaetano-Ortlieb , A. Kazantseva , N. Reiter , and S. Szpakowicz . Punta Cana, Dominican Republic (online): Association for Computational Linguistics , Nov. 2021 , pp. 128 - 137 . doi: 10 .18653/v1/ 2021 .latechclfl- 1 .1. 5

[40] T. Sommerschield , Y. Assael , J. Pavlopoulos , V. Stefanak , A. Senior , C. Dyer , J. Bodel , J. Prag , I. Androutsopoulos , and N. De Freitas. “ Machine Learning for Ancient Languages: A Survey” . en. In:Computational Linguistics 49 .3 ( Sept . 2023 ), pp. 703 - 747 . issn: 0891 - 2017 , 1530 - 9312 . doi: 10 .1162/coli_a_ 00481 . (Visited on 10/20/ 2024 ).

[41] T. Sommerschield , Y. Assael , J. Pavlopoulos , V. Stefanak , A. Senior , C. Dyer , J. Bodel , J. Prag , I. Androutsopoulos , and N. de Freitas. “ Machine Learning for Ancient Languages: A Survey” . In:Computational Linguistics (Sept . 2023 ), pp. 703 - 747 . doi: 10 .1162/coli_a_ 0 0481 .

[42] L. Spengel . Rhetores Graeci . Vol. 1 . Teubner , 1885 .

[43] R. Sprugnoli and M. Passarotti , edPsr. oceedings of the Third Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA) @ LREC-COLING-2024 . Torino, Italia: ELRA and ICCL, May 2024 .

[44] R. Sprugnoli , M. Passarotti , F. M. Cecchini , M. Fantoli , and G. Moretti. “ Overview of the EvaLatin 2022 Evaluation Campaign” . InP:roceedings of the Second Workshop on Language Technologies for Historical and Ancient Languages . Ed. by R. Sprugnoli and M. Passarotti . Marseille, France: European Language Resources Association, June 2022 , pp. 183 - 188 . url: https://aclanthology.org/ 2022 .lt4hala- 1 . 29

[45] G. Storey and D. Mimno . “ Like Two Pis in a Pod: Author Similarity Across Time in the Ancient Greek Corpus” . en. InJ:ournal of Cultural Analytics 5.2 (July 2020 ). issn: 2371 - 4549 . doi: 10 .22148/001c. 13680 . (Visited on 10/20/ 2024 ).

[46] M. Sundararajan , A. Taly , and Q. YanA. xiomatic Attribution for Deep Networks . 2017 . eprint: 1703 .01365. url: https://arxiv.org/abs/1703.01365.

[47] G. Thiele. “ Dionysii Halicarnasei quae fertur ars rhetorica rec. Hermannus Usener” . In: Göttingische Gelehrte Anzeigen 159 ( 1897 ), pp. 237 - 43 .

[48] J. Tyo , B. Dhingra , and Z. C. LiptonO.n the State of the Art in Authorship Attribution and Authorship Verification . 2022 . arXiv: 2209 .06869 [cs.CL]. url: https://arxiv.org/abs/220 9.06869.

[49] H. Usener . Dionysii Halicarnasei quae fertur Ars Rhetorica . Latin. Leipzig: Teubner, 1895 .

[50] H. Usener and L. Radermacher , edsD. ionysii Halicarnasei quae exstant . Vol. 6 : Opuscula, volumen secundum. grc . Vol. 6 . Stuttgart-Leipzig: Teubner, 1929 .

[51] A. Vaswani , N. Shazeer , N. Parmar , J. Uszkoreit , L. Jones , A. N. Gomez , L. Kaiser , and I. Polosukhin.Attention Is All You Need . 2017 . url: https://arxiv.org/abs/1706.03762.

[52] C. Walz .Rhetores Graeci. 1834 .

[53] K. Weismann. De Dionysii Halicarnassei vita et scriptis: Diss. inaug. Steuber, 1837 . url: https://books.google.nl/books?id=5XJSAAAAcAA.J

[54] I. P. Yamshchikov , A. Tikhonov , Y. Pantis , C. Schubert , and J. JostB. ERT in Plutarch's Shadows . Nov . 2022 . url: http://arxiv.org/abs/2211.05673(visited on 12/29/ 2022 ).