As the most proli昀椀c of the Church Fathers, John Chrysostom (344-407 CE) has a vast textual mass and theological importance that has led to a signi昀椀cant misattribution of texts, resulting in the existence of a second corpus known as the pseudo-Chrysostomian corpus. Like many Greek-language Church Fathers' works, this corpus comprises anonymous texts, which scholars have attempted to reattribute or group together based on factors such as the person's function, biography, ideology, style, etc. One survey conducted by Voicu in 1981 explored potential groupings of such texts and produced a critical list of 21 Pseudo-Chrysostom works identi昀椀ed by scholars, including Montfaucon (1655-1741), one of the 椀昀rst modern editors of Chrysostom's writings. In this paper, we present a novel approach to addressing pseudonymous work in the context of Chrysostomian studies. We propose to employ Siamese networks within an authorship veri昀椀cation framework, following the methodology commonly used in recent computational linguistic competitions. Our embedding model is trained using commonly used features in the digital humanities landscape, such as the most frequent words, a昀케xes, and POS trigrams, utilizing a signal-to-noise ratio distance and pair mining. The results of our model show high AUCROC scores (84.5%). Furthermore, the article concludes with an analysis of the pseudo-Chrysostoms proposed by Voicu. We validate a signi昀椀cant portion of the hypotheses found in Voicu's survey while also providing counter-arguments for two Pseudo-Chrysostoms. This research contributes to shedding light on the attribution of ancient texts and enriches the 昀椀eld of Chrysostomian studies.
Late Antiquity literature in Latin and Ancient Greek bears a profound in昀氀uence from Christian literature, shaping the era’s literary landscape. Patristic studies focus on the examination of the earliest Christian authors, spanning from the 1st century CE to the 7th century CE, with some scholars extending the period up to Jean Damascene in the middle of the 8th century ‡ § Contribution on all domain. Contribution on the pseudo-chryostomian context and evaluation of results in regards to the “traditional” patristic academic work. CE. Within the realm of Church Fathers, two prominent 昀椀gures stand out for their enduring impact on the corpora: Augustine (354–430) in Latin and John Chrysostom (344–407) in Ancient Greek. Despite their signi昀椀cance, both Augustine and John Chrysostom encountered a common challenge in Late Antiquity. Numerous works were falsely attributed to them, possibly with the intention of elevating the popularity of these writings, whose actual authors were less renowned. This misattribution has led to the creation of an extensive body of texts with uncertain authorship. According to Voicu39[], there are “more than a thousand” pseudoChrysostomian works whose true originators remain unkno1w.Tno address this issue, scholars of patristics have attempted to categorize some of these spurious works, associating them with a single person, who may be anonymous or identi昀椀ed with a speci昀椀c historical 昀椀gure.
The task of distinguishing anonymous authors within the pseudo-Chrysostomian corpus is
a signi昀椀cant and essential endeavor. By doing so, historians, patristicians, and philologists can
work with a more manageable collection of texts. This process o昀ers valuable insights into
the construction of Christian theological frameworks, allowing the identi昀椀cation of
ideological or thematic clusters within authorship groups. Establishing authorship for texts with at
least one proposed date enables scholars to study the evolution of Christian ideology more
accurately. Moreover, in rare cases, dating or attributing a speci昀椀c anonymous text (especially
when milestones are present) can provide new dates to other texts, as they might be cited or
reused [
In traditional patristics, authorship attribution largely relies on the identi昀椀cation of rare
patterns [
In such a context, automatic authorship veri昀椀cation can o昀er new insights that either
support or challenge previous hypotheses. Two other 昀椀elds that are actively engaged in authorship
veri昀椀cation are computational linguistics and digital humanities (DH). In the realm of
stylometrical analysis, DH scholars tend to treat texts as bag-of-words and utilize statistically signi昀椀cant
features, such as the most frequent words of a corpus9][, part-of-speech 3-grams, and
character n-grams [
In our research, we propose a novel methodology that combines approaches from both computational linguistics and digital humanities to analyze the 21 potential Pseudo-Chrysostoms 1Voicu has been working on a database for these textsh,ttps://www.trismegistos.org/pseudo-chrysostomic.a/ identi昀椀ed by Voicu and his predecessors. Speci昀椀cally, we aim to evaluate the e昀ectiveness of di昀erent features, including most frequent words, a昀케xes, and POS 3-grams, within the context of a Siamese network using a linear projection in N dimensions. To maximize the potential of our corpus, we introduce Easy-SemiHard Pair Minin34g][ to our batch learning process and utilize a signal-to-noise ratio distanc4e3[] to distinguish and separate our texts.
In summary, the contributions of our paper are as follows: • Introducing a new approach to authorial veri昀椀cation for ancient texts, incorporating Siamese networks and easy-semihard pair mining into the landscape of stylometry within computational humanities. • An analysis of various size for three types of features (POS, MFW, A昀케xes), which shows consistency in terms with previous studies. • An evaluation of common distances (Manhattan and Euclidian distance [L2]) and a newly introduced one (signal-to-noise ratio distance), which shows that the latter out-performs the best performing Manhattan distance in terms of stability within our settings. • Conducting an in-depth analysis of the results obtained from applying our approach to the 21* pseudo-Chrysostoms’ texts identi昀椀ed by Voicu.
The remaining of the paper is organised as follows. Section 2 provides background on the article of Voicu and in general on the issue of the pseudo-Chrysostomian corpus, as well as a deeper background on authorship veri昀椀cation and stylometry in general. Section 3 (Proposed Methods) provides details about the architecture used for the experiments. Section 4 (Experimental Setup) provides insight on the corpus, features selection and metrics. Section 5 provides an evaluation of the results on independent test sets. Section 6 reuses the models built and to provide insight on the PC corpus.
Regarding the pseudo-Chrysostomian corpus, a non-specialist might notice the lack of recent discussions concerning various attribution hypotheses. Most of these ”old” hypotheses trace their origins back to the 18th century, thanks to the diligent e昀orts of Bernard de Montfaucon, a Benedictine monk who dedicated his work to publishing the works of Athanasius (1698) and John Chrysostom (1718). Montfaucon’s editions and commentary laid the groundwork for later comprehensive editions of numerous patristic tex3ts0][, including the renownedPatrologia Graeca compiled by Migne, which is still widely used today as it is now in the public domain. Fortunately, in 1981, Voicu3[9] provided the most recent comprehensive summary of the authorship hypotheses within the pseudo-Chrysostomian corpus. This summary encompassed both refutations and ongoing debates surrounding various authorship attributions. Since then, some new hypotheses have emerged, but there hasn’t been a comparable e昀ort to Voicu’s in terms of summarizing the existing scholarship.
In his paper, Voicu provides a summary regarding authorship clustering of 88 texts, grouped under the pseudo-identity of 21 di昀erent authors by various scholars (PC1 to PC21). On top of the 21 base PC, another group of text is hypothetically attributed to one of them: PC20 is accompanied by PC20b, for which he dra昀琀s a possible ensemble without con昀椀rmation that they might be of the same authors (see Table1). The arguments regarding the attributions can vary, and we summarise them in four di昀erent categories: • Theological arguments are based on ideological (in)compatibilities between texts, in the context of a non-uni昀椀ed Christian religion in which subgroups can be foundE. .g. PC16 is categorized as a moderate Antiochian, which makes them incompatible with an Alexandrian ideology. • Sequential arguments are based on the obvious continuity between two texts, with established narrative links in both senses (Text A builds on Text B and vice-versa). • Stylistic arguments are based on the style of the authors. They range from the use of what is perceived as ”bad Greek” (PC6 and 7) to citation habits regarding the scriptures. • Extra-textual arguments are mostly based on events or texts referenced within the texts, which gives them a commonterminus ante quem orpost quem, or actually makes them incompatible. It also refers to transmission proofs, such as the constant grouping of the same texts in their transmission history (PC17).
Most of the cited texts are available in digital formats, speci昀椀cally in thTehesaurus Lingua Graeca, which is unfortunately closed acce2s.s However, one PC could not be tested in our 2All the feature extraction process is shared within the repositories, and the features themselves are made available. framework: PC19 only refers to one text in Ancient Greek, the other text being a Syriac translation. PC12 has been produced using the OCR available on Google Books, which has been lightly post-corrected.
The present work is related to two 昀椀elds: computational linguistics (CL) and digital humanities (DH). The existing literature reveals distinct approaches to the problem of authorship veri昀椀cation or authorship identi昀椀cation, primarily in昀氀uenced by the attributes of each 昀椀eld’s corpus and expectations for explainability. We focus on the latest approaches applied in both 昀椀elds, emphasizing the common technical approaches they have shared in the past.
Computational Linguistics CL o昀ers a clear landscape thanks to PAN, a ”series of scienti昀椀c
events and shared texts on digital text forensics and stylometry.” PAN’s shared tasks have
provided recurring competitions since as early as 2011, focusing on authorship attribution (2011,
2012, 2018, 2019) or veri昀椀cation (2013–2015, 2020–2023). As in most other computational
linguistics tasks, deep learning has seen a rise in popularity, leading to improved scores but lower
explainability. An insight into some of the approaches taken in authorship veri昀椀cation since
2015 reveals the following methods and features used:
• In 2015, Word-based n-grams, sentence length, word frequencies, punctuation
frequencies, POS frequencies, and POS n-grams were shared features across di昀erent paper2s8[,
27]. These features were eventually fed into standard classi昀椀ers such as Random Forest
or SVMs.
• In the 2018 PAN authorship attribution task1[9], features continued to focus on
characters and word n-grams, with various weighting and normalization methods. While
SVMs were commonly used, some early neural network approaches also appeared.
• In the 2020 PAN authorship veri昀椀cation task [16], methods employing neural networks
made an appearance, particularly in the form of Siamese neural networks. Features
mainly remained classical stylometric features, such as normalized frequencies of tokens
or POS. The winning paper utilized a Siamese network with extracted ”linguistic
embedding vectors” using an LSTM network with attention to produce document embeddings.
• In 2021 [17], features and approaches from 2019 were carried over, with a paper using
BERT and another one using Siamese networks ranking 昀椀rst and second place,
respectively, in the large dataset competition on overall scores, with the 昀椀rst one winning the
competition on all provided metrics.
• In 2022 [
With this summary, we observe that CL is gradually moving away from explainability and
increasingly employing text sequences (using RNNs, CNNs, or transformers like BERT or T5)
rather than treating texts as bags of words (BoW). Scoring is conducted using various metrics,
including AUROC and a modi昀椀ed F1-Score 0.5 [
In contrast to previous competitions, 2022 departed from 2021 in the type of dataset used. While 2021 and previous shared tasks utilized datasets sharing a common domain (such as fan椀昀ctions for 2021), 2022 focused on cross-discourse authorship veri昀椀cation , encompassing ”emails, essays, texts messages, and business memos.” This shi昀琀 resulted in a signi昀椀cant performance drop compared to previous years. Additionally, regarding the dataset, each competition relied on 昀椀xed pairings of texts, with limited or no information about the author in the metadata of the documents.
Digital Humanities Unlike CL, DH has largely preferred using feature selection and treats
most texts in an authorship attribution framework rather than authorship veri昀椀cation. The
three most frequently recognized features are: most frequent wor8d,s3[
In terms of technologies, tools o昀ering explainability (particularly feature weight) are highly favored, particularly SVMs or, in an unsupervised setting, hierarchical clustering using distances such as Manhattan, Cosine, Burrows’ Delta4[], or Eder’s Delta1[0], etc. We hypothesize that the need for explainability arises due to the fundamentally di昀erent objectives pursued by computational linguistics (CL) and digital humanities (DH) in the context of authorship veri昀椀cation and identi昀椀cation. While CL is primarily concerned with developing robust models at scale, DH seeks to employ the results as potential evidence in scholarly research. Consequently, the ability to interpret the decisions made by a model becomes invaluable for detecting ”invisible biases to the human eye,” such as the presence of unique characters in speci昀椀c author editions, which might lead the model to over昀椀t and yield erroneous conclusions. This risk is signi昀椀cantly ampli昀椀ed in the context of the frequent diachronic peculiarities prevalent in most DH inquiries or the constraints posed by a limited pool of available historical documents. Conversely, CL typically concentrates on short-term, cohesive, and large corpora, such as social network messages or fan 昀椀ction, where interpretability might not be as crucial.
In 2016, General Imposters (GI) methods2[0], an authorship veri昀椀cation method, were
introduced to DH 1[
We propose to bridge the gap between DH and CL practices by introducing supervised authorship veri昀椀cation using text pairing validation through a siamese network. However, Voicu’s anonymous PCs should not be used in the context of supervised training and authorship attribution. This is because supervised training relies on having ground truth data to identify the anonymous authors, and for some of Voicu’s PCs, they may not even be processed through the general impostors method as they only o昀er a single pair of texts, which is insu昀케cient for e昀ective training and veri昀椀cation.
Features Following most of the DH literature and PAN approaches until 2020, we select the relative frequency of most frequent words (MFW), POS tri-grams (MFP), and trigram character a昀케xes (MFT) within each text. We reject the use of any punctuation-based features, as punctuation in our texts is the result of the editorial task and can be editor or period dependent (an editor from the 18th century might not punctuate like a contemporaneous one). In order to test these, we tested a range of di昀erent values for each features: 1. 0, 100 and 200 most frequent POS-trigrams 2. 0, 250, 500, 750, 1000 most frequent word4s 3. 0, 250, 500, 750, 1000 most frequent a昀케xes.
MFP(100), MFW(1000), MFT(1000) provided the lowest standard deviation and highest AUCROC with two di昀erent distances (cf. Appendix, section B). These numbers, at least for MFW, are in line with current literature, most notably the recent paper from Rebor31a]’s. [We concatenate the relative frequencies of these features into a single vector, denoted as feature,s consisting of 2100 dimensions representing the text sampl e.
Model structure The model follows a typical architecture of linear layers in the context of Siamese networks (see Figure1): feature vectors are passed through the same projection layers, paired, and a distance-based decision is proposed. We use linear layers to red utcoe a given dimension , resulting in an embedding representatio n of the text. For the loss and distance metric, we tested Manhattan, Euclidean (L2) and “Signal-to-Noise Ratio” distanc4e3[] (SNR, SNRD for the distance) with contrastive loss. SNR-D considers the noise between an anchor embedding and a compared embedding , represented as = − , and uses its variance as an informative measure (variance of values across dimensions). The SNR for pair , is de昀椀ned as , = (( )) , and the SNR-Distance is SNRD = SN1R . Unlike distance metrics such as the Euclidean or Manhattan distance, SNR D≠ SNRD , which can provoke unmirrored attributions (cf. our analysis of PC corpora).
Learning methods Unlike PAN tasks, we can build our own corpus without pre-established
pairs, allowing us to utilize pair mining methods, which are particularly useful in the context of
Siamese networks (see Figure2). We use Easy–Semi-Hard (ESH) triplet mining4[
Datasets Unlike social networks content or fan-昀椀ctions, or even 19th-century literature,
Ancient Greek literature spans from 9 BCE to at least 9 CE in the context of our problem. We are
dealing with variation in corpus genre, linguistic changes, ideological changes, etc. In order to
address this at the training and evaluation step, we design a corpus that is focused on Christian
and theological texts using the TLG CD-ROM2[] and their XML export through Diogenes1[
Within the available corpus, we removed: • any dubia orspuria (de-attributed texts) or anonymous texts; • any text from John Chrysostom, as the sheer mass of his work and his style seemed to impact the model too much in early experiments, as well as both tOhled and New Testaments; • texts marked as commentaries,scholia, fragments, codices transcriptions, or speci昀椀crecensio; • any text containing a single line of poetr6y.
However, all of our most frequent features were extracted from the full Christian corpus.
To counter the reduced mass of texts and address the imbalanced state of the dataset, we produced 1000-word samples for each text using the following rules: • if the text was shorter than 2000 words, we used the 1000 words in the middle of the text, in order to avoid introductions and conclusions. • If the text is larger than 2000 words, we used up to 5 samples of 1000 words, randomly selected within the text except for the 昀椀rst 500 words and the last 500 words.
We then split the corpus in an 80-10-10 ratio along the titles of the various works, so that authors can span in di昀erent sets but samples of the same title remain inside the same text (see Figure3). The 昀椀nal corpus is diverse in genre but is composed of two genres which make up around two-thirds of its texts (Homelia and Treatise) while Oratio, Letters, Hagiography, and other non-frequent genres rapidly decrease in numbers of titles (see Fig4u,rfuell list of authors in appendix, TableD).
The PC corpus was not sampled, but relative frequencies were issued using the same process. Metrics We chose to evaluate the general performance of the model based on the Area Under the Curve of the Receiver Operating Characteristics (AUCROC), which allows for measuring the relationship between the False Positive Rate (FPR) and the True Positive Rate (TPR). This metric also enables us to select a task-oriented threshold by minimizing the FPR while maintaining a high enough TPR.
Evaluating stability Adding to this metric, we chose to also evaluate the stability of the
precision, a昀琀er seeing in our 昀椀rst experiments important variation between models using the
same parameters. We expect the randomness of training neural networks to be responsible for
producing latent spaces focusing on di昀erent features, and as such wanted to provide a full
picture of these potential variation. To evaluate the variation, we evaluate
5We hope, however, that the Patristic Text Archive3[
1. the standard deviation of their AUCROC, independently from the other hyperparameters.
Each models was trained three times for each parameters combination, we look at their
average standard deviation across distances.
2. using the framework of inter-annotator agreement through the Fleiss Kap pa, F(leiss
and Cohen [
Qualitative analysis method and pairing percentage To apply our model to the pseudoChrysostomian corpus, we need to provide a con昀椀dence score to interpret the distance between pairs of texts. We choose to compute the precision of our model at a given distance threshold
for each pair A-B (and its reverse B-A) on both the development set and the test set. This threshold produces a “precision threshold” (PT) that allows us to estimate the probability of a false positive at any given distance. For example, i f for distances less than or equal to 0.4 is 0.9, the n = 0.4 and the precision of the model = 0.9, indicating a high probability that A and B are from the same author.
In order to have a quick overview for the pseudo-Chrysostomian corpus as a whole, we evaluate the “pairing percentage” at a variety of precision threshold. Pairing percentage indicates how many positive pairing are operated at a given PT.
Hyper-parameters optimizer, learning rate o1f −4
So昀琀ware implementation
The experiment was implemented using the following so昀琀ware
and hardware: it was run on an nVidia RTX 3090 GPU with 24GB of RAM, usintgorch [29],
torchmetrics [26] for AUROC metrics,pytorch-lightning [12] for training,
andpytorchmetric-learning [
− AUCROC dev and test sets (cf. Table2b).
On top of the features selection, unlike most traditional DH approaches, training linear models to produce latent spaces can introduce potential randomness due to the initialization of model weights, the choice of optimization algorithms, etc. This possibility is exacerbated by the fact that we are using a changing development pool produced through ESH mining of the full development set.
Looking at the standard deviation of our models across all parameters (T2aab)l,eSTN seems to have the most stable standard deviation moving from dev to test, and the lowest on the test set. This would indicate that STN based model are more stable across runs from a score perspective, and that, at least in our context, that its performances translate in a similar fashion from the dev to the test set. This is further con昀椀rmed by our analysis of the variation of across our parameter search, with a very stable distance between
Looking at our models randomness further, we can see that the models using STN distance (Table3) are much more in agreement across 100 di昀erent models and training seed than those using Manhattan distance, despite higher AUCROC scores. We can see that STN has more than 2.5 times the of Manhattan, and Manhattan only reaches the . 5 at 80% of PT. In a philological set-up, this advocates even more for the use of the most stable model.
Looking at the qualitative dataset and pairing percentage, we can see the same phenomenon, with much more unstable pairing using Manhattan rather than STN (Figu5r)e. At 100 PT, the (a) pMaeraanmoeftesrtasnsdwaredep-ddeveipaetniodninagcroonssdtihse-(b) Absolute di昀erence between dev and test scores across tances. models deMpeeannding±on tmhien dis2t5a%nceMuesdeiadn. 75% max standard deviation is 50% for more 6 clusters across for the Manhattan distance, a number unseen using Signal-to-noise at the same threshold. Only lower PT achieve such a high standard deviation (See appendix for more 昀椀gures across 100 models).
The results achieved in this study fall within the range of the current state of authorship
veri昀椀cation. Tyo, Dhingra, and Lipton [
CROC7. The proposed model stopped converging a昀琀er 219 epochs on the ESH mined pairs and provides an 86.75% AUCROC on all pairs of the development set (see Tab4l)e. The model reaches a slightly lower score on the test set with an 83.76% AUCROC. This is inline with the oberserve di昀erences during the parameter sweep. We also would like to note that due to ESH mining, the pairs used in the dev set for loss might change from one iteration to the others, as the embeddings are used to decide which pairs to use, resulting in a di昀erent kind of over-昀椀tting than with a traditional 昀椀xed development set.
(a) Development set (b) Test set
The ROC curves (see Figure6) exhibit a compelling shape, with a signi昀椀cant slope in the initial percentages of the False Positive Rate (FPR), followed by a more gradual increase until reaching 100% True Positive Rate (TPR). This suggests that the model e昀ectively distinguishes between positive and negative pairs, especially in the early stages of classi昀椀cation. UMAP22[] (Figure7), while still serving as a proxy for the complexity of the embeddings’ dimension, provides a clear depiction of the discrepancy between both sets. In the test set, embeddings of texts from the same authors are notably more distant from each other, indicating that the model’s representations are better disentangled in this set. On the other hand, the development set displays more overlap and mixing of embeddings, suggesting that the model’s representations 7Note that our corpus are di昀erent. Numbers from Tyo et al. are given to provide some form of context, given the lack of application of this method on corpus similar to ours. are less distinct and might have captured some biases from the data splitting process.
Using a visualization of pairing percentage using precision threshold (Precis≥ion0.75, see Figure8), we see that our overall results are remarkably consistent with the summary proposed by Voicu. Notably, the PC 3, 4, 5, 8, 9, and 13 are not con昀椀rmed as single authors by our model and are either indicated as low probability clusters or refuted groupings in Voicu’s article. On the other hand, only the 20 and 20b clusters are completely ignored by our model despite having a high chance of being from the same authors according to both Datem7a][and Voicu. Among the highly connected clusters, PC 1,82, 12, 16, 17, 18, and 21 are all marked as highly possible or con昀椀rmed by Voicu and are found at a precision threshold of 85% or higher, either on dev or test or both.
We turn our attention to the pseudo-Chrysostoms whose scores are not relating to the philological studies: PC 6–7, 10–11, 14–15, 20, and 20b.
Montfaucon’s pseudo-Chrysostoms 6 and 7 Both pseudo-Chrysostom 6 and 7 are hypotheses made by Montfaucon in the 18th century. PC6 is considered to be written by bad (and stupid) ancient Greek speakers i(nepti Graeculi), with ad nauseam use of repetitions and epithets, while ignoring common rules of grammar (PG, 60, 681-682). Regarding PC7 and Dthee jejunio sermons, the only arguments of Montfaucon (PG, 60, 711-712) were again his absence of knowledge of basic Greeki(mperiti Graeculi) and his ability to write mostly nonsensical things (plerumque nugacis).
Looking at the precision threshold matrices in Figu9r,ewe see that the matrix behaves in a non-reciprocating way, such that some texts are deemed to be of the same author in the sense A–B but not in the sense B–A. Moreover, for PC7, we see that consecutive sermons are unconnected between each other e(.g. De jejuno 3 and De jejuno 4 are not deemed to be of the same author at any precision) but are connected with subsequent or previous sermoen.gs.(De jejuno 3 and De jejuno 4 are deemed to be of the same author asDe jejuno 5). Given the scores, we might be tempted to con昀椀rm a single author for each of the PC6 and PC7 clusters, but we cannot con昀椀rm this without reasonable doubt. Our hypothesis, though untested, suggests that if PC6 and PC7 employ such an unconventional form of Ancient Greek compared to the majority of the corpus, they could prove challenging to categorize uniformly. The case of Pseudo-Chrysostoms 13, 14, and 16 PC13, 14, and 16 are a particular case in Voicu’s survey, as PC13 is refuted by Voicu and its texts are dispatched into two other subcorpora: two texts C(ontra Iudaeos, Gentiles et Haereticos and In uenerabilem) are thought to go with PC14’s De Eleemonsyna, whileDe Epiphania would go with PC16. Our precision matrix threshold does con昀椀rm the unity of PC16 except for the inclusion of PC13’s text (see Figure 11a). However, while the pairs connections are low for PC13 and PC14, they seem to be rather 8The situation is less clear for PC2. It seems that it is well detected by our Manhattan models,1s3e.e (a) Pseudo-Chrysostom 6 (b) Pseudo-Chrysostom 7 high when connected to PC16. Our model seems to recommend the following composition for PC16: • De Eleemosyna (PC14) • In Psalmum Homilia 50 homiliae, 1–2 (PC16) • In Illud: Su昀케cit Tibi Gratia Mea (PC16) • In Illud: Si Qua Christo Nova Creaturqa (PC16) • (Lower probabilityC)ontra Iudaeos, Gentiles et Haereticos (PC13*) • (Lowest probability)In Venerabilem Crucem (PC13*) Pseudo-Chrysostom 20 and 20b: What happened? The most signi昀椀cant disagreement between traditional scholarship and our model arises with PC20, which is classi昀椀ed as uncon昀椀rmed by our model (see Figure10). Voicu mentions PC20 as the outcome of ”on-going research” and comprises 6 base texts (PC20) and 12 potential others (PC20b). Unfortunately, some of these texts are unedited, and we did not have access to them. As of now, none of the texts in PC20 and PC20b form prominent clusters.
We can see two pairs that might be interesting to focus on, speci昀椀cally: • 0.91 In Drachmam ...–In Parabolam..., with a .86 PT in the other direction. Both are within a high range of PT and comes from the same PC corpus; • In Illud: Ascendit and In Parabolam de Ficu, with .60 and .74 scores.
However, the connection betweenIn Drachmam and In Illud: Ascendit is not strong (.3).
Moreover, the precision thresholds for most pairs with high values are not reciprocated when considering the inverse direction: • 0.82 In Rachelem...–De Non Judicando, with a 71-point drop in the other direction;
• 0.85 In Rachelem et in Infantes–Quod Mari Similis, with a 75-point drop in the other direction.
The lack of clear clustering and the signi昀椀cant di昀erences in precision threshold values between pairs in opposite directions raise questions about the coherence and authorship attributions within PC20 and PC20b. It is essential to recognize that authorship veri昀椀cation is a complex task, and discrepancies between our model’s 昀椀ndings and traditional scholarship can be attributed to various factors, including the linguistic features used for analysis, the dataset’s size and quality, and the nature of the texts themselves. We also want to stress that we have a low recall, which can lead to having many false negatives.
Others For other Pseudo-Chrysostom, we propose the following analysis:
• Pseudo-Chrysostom 10 (Figure11c): This cluster falls into the group of
PseudoChrysostoms for which the A–B and B–A distances di昀er. As it consists of a single
pair of texts, and that the con昀椀dence is strong in at least one direction, we would be
inclined to con昀椀rm the assumption of Weyer[
according to Nautin2[
Authorship veri昀椀cation is a crucial task in the 昀椀eld of computational humanities and the humanities in general, as it o昀ers a new approach to validate older hypotheses made using traditional philological methods, such as transmission study or stylistic analysis. This is particularly important in patristic studies, where pseudo-author corpora, such as those attributed to Augustine or John Chrysostom, are signi昀椀cant. The challenge lies in dealing with pseudonymous corpora where most authors are unknown.
To address this, we proposed exploring the use of Siamese Networks with embeddings based on known stylometric features, such as the relative frequencies of most frequent words, a昀케xes, and POS 3-grams. To optimize our approach, we leveraged pair mining and signal-to-noise ratio distance, both originally designed for Siamese network architectures. The results we obtained on our development and test sets are very promising, and we argue that, unlike authorship attribution problems, authorship veri昀椀cation problems are less susceptible to over昀椀tting when using such architectures.
Our 昀椀ndings mostly align with the survey conducted by Voicu in 1981, which identi昀椀ed 21 potential unique pseudo-authors in the corpus of the pseudo-Chrysostomian texts (see Table 5). Some pseudo-Chrysostoms are highly probable under both the framework of classical philological studies and our approach. However, we encountered discrepancies with two pseudoChrysostoms. First, PC1, hypothesized by Montfaucon and refuted by Altendo1r]f,[is mostly con昀椀rmed within our framework. Second, we were not able to con昀椀rm PC20. It’s important to note that our method is designed for precision rather than recall, so this result does not necessarily refute Voicu’s hypothesis but still stands out compared to the rest of our results.
In future work, we are interested in extending our approach without relying on a learned embedding space and instead using probabilistic tools for better explainability. The instability of the models we saw with Manhattan distances and the lower – yet existing – instability of STN based models argue for more interpretable and stable models. This kind of work has already been explored by Weerasinghe, Singh, and Greenstad4t0[] and would be a valuable addition to the digital humanities landscape. Further analysis on PC20 should be produced, speci昀椀cally by looking at the research produced by Voicu since 1981 on this particular topic.
Code and data for the Voicu experiment are availablehatttps://github.com/PonteIneptique/ Chryso-Voicu.
Features and parameters search and their respective AUROC (in %). Each con昀椀guration is run three times (seed 42, 128, 256), average and standard deviation are provided. Tested parameters: MFP (0, 100), MFW (0, 250, 500, 1000), MFT (0, 250, 500, 1000), Distance (Manhattan, L2 and STNR). Scores are ranked according to their test AUROC. Only the top 10 are shown, the remaining scores are on the online github repository.
POS FW
Parameters
A昀케xes Distance
(a) Using Manhattan (b) Using Signal-to-Noise Ratio distance Figure 12: Mean of the pairing percentage in clusters depending on the PT.
(a) Using Manhattan (b) Using Signal-to-Noise Ratio distance Figure 13: Median of the pairing percentage in clusters depending on the PT. 245