The literary sources behind the three canonical Synoptic Gospels, namely Luke, Matthew and Mark, have long intrigued scholars because of the Gospels striking similarities and notable di昀erences in their accounts of Jesus's life. Various theories have been proposed to explain these textual relationships, including common oral witnesses, lost sources or communities possessing each other's works. However, a universally accepted solution remains elusive. Leveraging advancements in statistics, data analysis, and computing power, researchers have begun treating this as a statistical problem and quantitatively measuring the likelihood of the di昀erent theories based on verbal agreements and stylometric features. In this paper, we rely on a very recent Machine Learning based approach to solve the synoptic problem. We use Machine Learning classi昀椀ers two-sample tests, a novel approach relying on the analysis of the success rate of binary classi昀椀ers to identify whether two samples are drawn from the same distribution, to detect di昀erences in sources within Luke's Gospel and variations in the edition patterns of Markan material between Matthew and Luke. This analysis is done on a pericope-per-pericope basis, de昀椀ned as thematic units encompassing teachings or narrative episodes. The results suggest signi昀椀cant dissimilarities in style and edit distance, indicating that the double and triple material within the Gospel of Luke likely originate from di昀erent sources. This suggests that Luke derived his triple tradition from Mark and not from Matthew. Despite the necessity of cautious interpretation due to the size of the dataset, our study thus o昀ers substantial evidence supporting the theory of Luke's dependency on Mark's material for his triple tradition and makes the two-source hypothesis, which suggests that Luke did not have access to Matthew's work, the most likely explanation based on our methodology.
The striking similarities, along with the notable di昀erences, between the di昀erent accounts of
Jesus’ life related in the three canonical Synoptic Gospels, usually referred to as the Gospel of
Luke (Lk), Matthew (Mt) and Mark (Mk), have led scholars since Antiquity to speculate on the
order of their composition and the literary sources available to each community the Gospels
came from. Over the years, several theories have been proposed to explain this phenomenon,
including the idea of shared oral witnesses8[], the existence of now-lost common sources [
In this paper, we suggest a novel approach based on the use of a Machine Learning classi昀椀er two-sample tests to detect di昀erent sources within the Gospel of Luke and di昀erent edition patterns of the Markan material between Matthew and Luke. We compare statistically the distribution of the stylometric features and show that the theory of Luke and Matthew being independent, also known as the two-source hypothesis, is the most likely given our methodology.
The major contributions of this study are: • The introduction of a novel paradigm by conducting stylometric analysis at the pericope level rather than focusing on individual verses when comparing the Synoptic Gospels. • The use of a very recent statistical approach, Machine Learning-based two-sample test, to identify stylistic di昀erences within double and triple material of Luke’s Gospel. • The application of this Machine-Learning based approach to characterize the edits made by Luke and Matthew when it comes to the Markan material.
The results of our study allow us to lean towards the two-source hypothesis as being the most likely one given the stylometric data of Luke’s Gospel.
This paper is structured as follow. We give in sectio2n more insights regarding the synoptic
problem and the major solutions suggested over the history of the Gospels. Sectio3npresents
works related to ours which rely on statistical methods to propose solutions to the synoptic
problem. Section4 presents our selected stylometric approach and its results when applied to
the SBL Greek New Testament (SBLGNT) text [
Solving the synoptic problem consists in providing a theory that describes the relationships
between the three canonical Synoptic Gospels (Matthew, Mark, and Luke) by analyzing their
similarities and di昀erences. If the potential in昀氀uence of oral traditions on early Christian
teachings cannot be overlooked [
Lk Mt
Lk Farrer-Goulder Hypothesis
Two-Source Hypothesis tradition (found in all three Gospels, o昀琀en with variations in order and context, such as the healing of the paralytic found in Mt 9:1–8, Mk 2:1–12, and Lk 5:17–26). The distribution of the di昀erent material per verse number and per pericope is available in tabl1e.
Traditionally, the Augustinian model3[] proposed that Matthew was written 昀椀rst, followed by
Mark as a summary of Matthew, and 昀椀nally Luke1. However, the past two centuries of textual
criticism have challenged this model, starting with Griesbach’s hypothesis29[] that Matthew
was the initial Gospel, which was later used by Luke, while Mark combined elements from
Luke and Matthew [
The use of statistical data in the analysis of the synoptic problem goes back to at least the end
of the nineteenth century, when Hawkins’Horae Synopticae was published in 1899 [
The research can be separated into two wide categories: the study ofverbal agreements [27, 1Followed by John. Due to the complexity and ongoing research in the 昀椀eld of the relationship between Johannine literature and the Synoptic Gospels, this article does not delve at all into this subject matter. 17, 7, 1, 21, 32], consisting in counting the number of occurrences of the same word being present in two Gospels within the same narrative framework, either in its in昀氀ected or lemmatized forms, and stylistic analysis of the Gospels for source detection. Much of the last 昀椀琀y years of research has relied on the study of these agreements and J.C. Poirier has provided a very thorough survey of the main verbal agreement work in31[]. He however concludes his study with little hope regarding the feasibility of relying on verbal agreements to ”昀椀nd a 昀椀nal solution to the synoptic problem”, as the ”statistical studies have too o昀琀en amounted to coded expressions of their user’s commitments”.
Another possible approach relies on the analysis of the style of the Gospels, either
usiCnogrrespondance Analysis as suggested by A. Linmans in [19] and D. Mealand in [23, 25, 24]. Notably,
Mealand provides several di昀erent approaches to leverage stylistic analysis to provide insights
regarding the likelihood of the 2SH: he uses in2[
Our contribution to the current state of the art of statistics and the Synoptic Gospels is threefolds : (1) we leverage more recent advances in data science and move on from Mealand’s and Linmans Linear Discriminant Analysis 2[4, 19] to classi昀椀er two sample tests using Random Forests; (2) we work at the pericope level instead of at the verse level or sliding pools of words; (3) we focus not only on stylometric analysis, but take into account the editing patterns of Luke and Matthew, which takes a step further the approach suggested by Linmans when he uses Principal Correspondance Analysis for parallels.
Data from the SBL Greek New Testament (SBLGNT) edition and its morphological parsing and
lemmatization, provided by MorphGNT 3[
To perform the stylistic comparison between the di昀erent pericopes, we decide to focus our study using only non-signi昀椀cant language patterns, instead of focusing on content word frequency with metrics liketf-idf computed on the corpus as a whole: as the pericopes relate the same stories, they naturally tend to use the same vocabulary which could bias a distance metric. To focus solely on style, we only keep features that do not bring any meaning to the sentence other than its grammatical and logical structure.
In total, 103 metrics were designed, based on recent advances in stylometry11[
The stylistic di昀erence between two pericopes is then de昀椀ned as the di昀erence between two stylometric vectors. The closer the style transformation, the smaller the di昀erences in each variable, and vice versa. For pericopes from the triple tradition, there are two distances per pericope, and from the double, there is one distance per pericope. Single tradition material does not have any distance, as it has no reference point within other Gospels by de昀椀nition. Stylistic di昀erence vectors for pericope number and feature will be denoted asΔ ( − ) , for , ∈ {, , }, ≠ . The gospel of Mark is used as a reference as both tested theory posits Markan priority.
For example, focusing only on feature 1 corresponding to the number of words in the pericope
of The Healing of Peter’s Mother In Law, which is within the triple tradition (Mt 8:14–15, Mk
1:29–31, and Lk 4:38–39, pericope 37 according to Aland’s classi昀椀cation), Mark uses 48 words,
while Luke uses 41 and Matthew uses 33: this results in a stylistic di昀erence for feature 1 of
Δ37( − ) 1 = −15, Δ37( − ) 1 = −7 and Δ37( − ) 1 = −8. This di昀erence is then
computed for every of the 103 stylistic features and we give an example of the computation of
these vectors in table2: for example, we can see that when editing theMother In Law Pericope,
Luke and Matthew both decided to reduce the number of proper nouns to 2 and to shorten the
number of words in the pericope. If similar editing choices are observed regularly, then this
would give evidence that editing was not performed independently and vice versa.
Because the reduced space can get sparse (as some pericopes have had little change), we choose
to apply Principal Component Analysis [
Working with such a large feature set poses a challenge due to the multivariate nature of the stylistic vectors, which would require multivariate response models if we were to work using a classic statistical framework: a potential solution, as suggested by Mealand in24[], is to employ Generalized Linear Models with a multivariate target variable. However, Mealand’s decided to limit its model to only 昀椀ve dimensions. Given the extensive input space in our study, such models would not provide reliable results as we would have to compute one statistical text per feature, leading to hundreds of statistical test that would increase the risk of family-wise error rate.
To tackle this issue and still work in a multi-dimensional space, we use the approach suggested
by Lopez-Paz et al. in [20], who suggest using the prediction of classi昀椀cation of Machine
Learning (ML) models as a two-samples hypothesis test in order to assess if two samples are from
di昀erent distributions, with the following heuristic: if a ML model can discriminate more
accurately between the observed samples than between randomly drawn ones, then the two classes
must have di昀erent distributions. More formally, we compare two samples, P and Q. If the null
hypothesis ( = ) is true, a binary classi昀椀er’s accuracy on a held-out subset should be around
chance-level. Conversely, if the alternative hypothesis≠( ) is true, the classi昀椀er’s accuracy
will be higher than chance-level, leading to rejection of0. As our classi昀椀er, we use Random
Forests [
To enhance the comprehensiveness of our study, we adopt a dual approach. We begin by examining the stylistic variations within the Lukan text between the double and triple traditions, aiming to identify distinct sources (as discussed in the next paragraph4.5.1). We then propose a novel approach by investigating the interdependence among the edits made by di昀erent authors (as outlined in paragraph4.5.2).
To decide which scenario is the most likely, one can inquire if the styles vary di昀erently across the triple and the double material: if so, one could assume that Luke has taken his double and triple material from two di昀erent sources, like the 2SH posits. It would however not completely invalidate the FGH, as one could argue that Luke could have taken all of his triple tradition material from Mark, but would mean that we cannot observe aneyditorial fatigue2 as argued by Goodacre in [12], where Luke style would match Matthew’s instead of following Mark’s. To assess the stylometric di昀erence across the sources, we design three di昀erent stylometric hypotheses to be evaluated using the test methodology described in subsection4.4. These tests are named using the conventionStylo.n, ∈ 1, 2, 3 and their alternative hypotheses and implications in the synoptic problem are summarized in tabl3e.
Stylo.0 measures style di昀erences between sayings and narrative in the triple and double tradition of Luke’s gospel to evaluate the relevance of our selected features. If we cannot detect signi昀椀cant stylometric changes, it implies our features may not be reliable for detecting subtle variations like distinct sources.Stylo.1 tests the style di昀erence between the double and triple traditions using a ML model. Successful classi昀椀cation would indicate that the style varies differently across the two traditions, supporting the idea that Luke sourced his double and triple material from separate origins, thus validating the 2SHS.tylo.2 acts as a sanity check, considering the genre of the classi昀椀ed contents. The goal is to build a model that accurately classi昀椀es 2As de昀椀ned by Goodacre in [ 13], ”Editorial fatigue is a phenomenon that will inevitably occur when a writer is heavily dependent on another’s work. In telling the same story as his predecessor, a writer makes changes in the early stages which he is unable to sustain throughout.” sayings common to Matthew and Luke, as well as those found in Mark, to ensure that the model does not oversimplify classi昀椀cation by labeling double tradition material as ”sayings” and triple tradition material as ”narratives.”
The other complementary approach that we suggest in this study is the comparison of the editorial choices made by Luke and Matthew of Markan material: if these patterns are di昀erent, then one could assume that they were made independently. We design two di昀erent statistical tests relying on a two-samples classi昀椀er,Edit.1 and Edit.2, that we summarize in table3. Edit.1 aims at comparing the distance between the triple tradition between the three Gospels in order to understand where Luke’s triple tradition comes from. If we make the hypothesis that a smaller variation in style means a similar source, then the distance between Luke-Matthew should be signi昀椀cantly closer than the distance between Luke-Mark over the triple tradition if Luke had access to Matthew. Otherwise, one can assume that Luke took his triple tradition from Mark, which would lead to weakening the FGH and be consistent with the 2SH, as Luke and Matthew are expected to perform their editing independentlyE.dit.2 aims at measuring the independence of Matthew’s and Luke’s use of Mark: the idea is to see if it is possible to distinguish an editing pattern between sources. If we can 昀椀nd no signi昀椀cant di昀erence from the way Luke edits Mark than the way Matthew edits Mark, one could assume that the edition process did not happen independently and Luke had access to Matthew’s triple tradition. These tests are then extended to narrative and sayings data, as sayings of Jesus could be expected to be less edited and are more likely to be takenverbatim.
The Random Forest classi昀椀ers, as well as the PCA, is taken fromsklearn [
The results of the stylometric testsS(tylo.n) and edit tests (Edit.n) are available in table4.
Stylo.0: Di昀erence in style between narrative and sayings material in Luke The test results demonstrate a signi昀椀cant ability of our stylistic features to di昀erentiate between narrative and sayings content within the entirety of Luke’s pericopes (p-value < 0.001). When employing a Random Forest classi昀椀er using 75% of the data as train and 25% as test, and strati昀椀cation based on observed genre, the classi昀椀er achieves an accuracy score of 81% on unseen pericopes (95% over the complete dataset). These results indicate that our stylistic features e昀ectively enable the classi昀椀cation of pericopes based on their style. When analyzing the misclassi昀椀ed pericopes, they tend to fall along the genre borderlines, with the 6 misclassi昀椀ed sayings containing many narrative elementsT( he Parable of the Rich Man and Lazarus, the Day of the Son of Man, On Riches and the Rewards of Discipleship, The Parable of the Good Samaritan, The Parable of the Wicked Husbandmen, The Pharisee and the Publican) and the 7 misclassi椀昀ed narratives featuring discourses (The Beelzebub Controversy, The Rich Young Man, John the Baptists’s Messianic Preaching, The Pharisees Seek a Sign, First Preaching Tour of Galilee, Peter’s Denial Prediction and The Third Prediction of the Passion). This high accuracy rea昀케rms the validity and relevance of the devised features for the remainder of our study. Stylo.1: Di昀erence in style between double and triple tradition material in Luke The two-samples test conducted on the classi昀椀er reveals a highly signi昀椀cant distinction (p-value of 0.03) between the styles of the double and triple traditions. Utilizing a Random Forest classi昀椀er with a test-train split ratio of 0.25, we achieve an accuracy of 91% for unseen pericopes and 97% across the entire dataset, successfully identifying material from both the double and triple traditions. This disparity in style strongly supports the hypothesis that Luke drew his material from two distinct sources for the double and triple traditions, aligning with the 2SH. While proponents of the Farrer’s model may argue that Luke derived the double tradition material from Matthew and the triple tradition material from Mark, this interpretation does not align well with the concept of editorial fatigue proposed by Goodacre, as our tests indicate that Luke disregarded Matthew’s style in his handling of the Matthean edition. However, the study of the confusion matrix available in tabl5eb shows that the unbalanced dataset a昀ects the accuracy per class (there are 127 triple tradition pericopes and 50 double tradition). Further works of ours will include adding some techniques for small unbalanced datas2e6t][, to enhance the quality of our prediction model.
shown in the previous paragraph, the stylometric features di昀er strongly depending on the genre of the pericope, and we need to make sure that the results of the tessttylo.1 are not only a measure of the di昀erence of style, as the double tradition is mostly composed of sayings. However, the results of testStylo.2, with a p-value of 0.05, shows a signi昀椀cant style di昀erence between sayings from the double and triple tradition, allowing us to con昀椀rm the stylistic difference between material, even when accouting for genre. A random forest using a test-train split ratio of 0.25 yields an accuracy score of 75% on unseen data and 92% of the whole dataset. The confusion matrix is available in table5c and the small sample size once again a昀ects the power of the evaluation of the classi昀椀er on test data: the error is however balanced across the double and the triple tradition.
reveal a statistically signi昀椀cant distinction (p-value of 0.02) between the stylistic similarity of Luke and Mark compared to Luke and Matthew. Luke and Mark exhibit a closer style (with a combined total distance of 50.46) than Luke and Matthew (with a combined total distance of 72.81). When a Random Forest classi昀椀er is trained using standard test/train 昀椀tting (with a 0.25 ratio of test and train split), it achieves an accuracy of 79% when classifying unseen samples and a 95% accuracy across the whole dataset. The signi昀椀cant proximity observed between Luke and Mark, and the distance from Matthew to Luke, suggests that Luke most likely drew his triple material from Mark rather than Matthew. This 昀椀nding once again contradicts the notion of editorial fatigue, as proposed by Goodacre, which would have resulted in a less pronounced di昀erence in stylistic distances. We believe it is thus unlikely that Luke relied on Matthew’s text to copy his triple tradition material, which supports the plausibility of the 2SH.
ducted to evaluate the variation between Matthew’s and Luke’s edits of Mark have yielded inconclusive results, with p-values greater than 0.05. However, it is worth noting that the pvalue of Edit2.1 is relatively low at 0.07, suggesting a tendency to reject the hypothesis that Luke and Matthew edit Mark in a similar manner. This inclination leads us to consider the hypothesis that Luke disregards Matthew’s edits and implies that Luke did not consult Matthew’s use of the Markan material. When examining sayings or narrative material, our study encounters limitations in drawing de昀椀nitive conclusions due to the combination of high p-values and low statistical power, particularly when the e昀ect size is small. The available dataset consists of only 156 triple tradition narrative pericopes and 96 limited triple tradition sayings pericopes. Consequently, our test subset comprises merely 39 and 24 samples, respectively, considering a 0.25 test-train splitting ratio. Given these constraints and until further works consisting on using methods for unbalanced dataset, we are unable to con昀椀rm or dismiss the notion of a difference in editing style between Luke and Matthew: we have to rely on the other performed tests, which all point to the 2SH.
The two facets of our analysis analysis seem to point towards the 2SH: the stylometric tests
show that the double and triple tradition di昀er signi昀椀cantly in terms of style, even when
considering sayings (Stylo.1 and Stylo.2). Even though one must remain careful on results on
smaller datasets, such a di昀erence seems to con昀椀rm that the double and triple material come
from di昀erent sources, and thus that Luke took his triple material from Mark and not from
Matthew. This use of Mark’s material by Luke is further con昀椀rmed by the distance stylistic
tests (Edit.1), which shows that Luke’s style is signi昀椀cantly closer to Mark’s than to Matthew’s.
Whenever comparing the editing style, the testEdit.2 tends to show that Matthew and Luke
use the Markan material di昀erently, even though the test p-value and power is not su昀케cient
to de昀椀nitely conclude regarding the di昀erence in material edition. We believe our features
aggregated on the pericope level might be too global and we will investigate in further works
more re昀椀ned features, such as the verbal agreements, the addition and the deletion orders and
use Deep Learning embedding models such as LSTM 1[
Whenever considering previous studies, our results generally align with results from other statistical studies of the synoptic Gospels: Mealand demonstrated (prudently) a di昀erence in word usage on Matthew’s Gospel into a Matthean, double, and triple tradition, and the Linear Discriminant Analysis he runs on Luke’s Gospel also shows a division in double and triple tradition. As there is to our knowledge no stylometric study of the edition of patterns between Matthew and Luke, we cannot compare our results to previous studies, but future works will compare the feature importance of the Random Forests to the manual works performed by scholars to characterize Matthew and Luke’s handling of Markan material.
As a conclusion, our work adds yet another argument towards the likelihood of the 2SH model. We however want to emphasize that our intention is not to o昀er a de昀椀nitive answer to the synoptic problem, but rather to introduce innovative analytical tools that can assist in presenting the most plausible solution to the problem through style analysis, even though numerous variables other than style, such as pericope orders, must be considered to provide a comprehensive answer.
Our further studies will include adding additional features better able to characterize source usage, such as verbal agreements, and see how it a昀ects the results of the edition tests. We will also work on augmenting the number of samples, either by simulation through bootstrapping or by considering moving windows across the di昀erent verses, to increase the statistical power of our study. We will also take into account variants instead of taking the SBLGNT as the 昀椀nal text of the Gospels.
This work was funded by the SCRIBES (Biblissima+ Grant) and the ANR SHERBET (CE38-2023) projects. [4] F. Baskevitch. Le vocabulaire du son en grec ancien - étude sémantique et étymologique.
2016.
D. Mealand. “Measuring Genre Di昀erences in Mark with Correspondence Analysis”. In: Literary and Linguistic Computing 12(4) (1997), pp. 227–245. [37] C. Weisse. Die evangelische Geschichte kritisch und philosophisch bearbeitet. Breitkopf und Hartel, 1838.
(b) Words with computed frequencies