Ming-Qing fiction is widely regarded as the pinnacle of classical Chinese literature, but over threequarters of vernacular fictional works were anonymously or pseudonymously composed, frustrating literary-historical research. To begin to address the problem, we propose a cross-register authorship attribution task: recover the authorship of a vernacular Chinese text given classical Chinese writing samples of known authorship. A corpus of eight authors known to have written in both registers was assembled to serve as a testbed. We describe the performance of models using diferent sets of function character/word frequencies as input features. This standard approach to authorship attribution performs well in the same-register setting but poorly in the cross-register setting. We discuss the degree of vernacularization and the amount of dialog in texts as key factors contributing to the low cross-register accuracy.
the previously mentioned factors. In practice, candidate authors are usually already provided
to researchers thanks to the labor of literary and cultural historians. The challenge lies in
matching writing styles. Numerous factors reportedly influence writing style, including genre
[
Classical Chinese can be understood as preserving the grammar and semantics of Chinese
as it was used before the Qin period (i.e., before 221 BCE). Classical Chinese predominates
in official texts and texts written by members of the educated class throughout the imperial
period (221 BCE - 1912 CE) [
Written vernacular often emerged from written dialog in classical works. This way of writing
developed into various genres. For example, Bianwen ( ) paraphrases canonical Buddhist
texts using speech-like writing. And Huaben ( ) describes actors’ movements and scripts
when performing. It was not until the middle sixteenth century before vernacular written
Chinese became a recognized literal register [
The diferences between the two versions of written Chinese are considerable. First, the classical lexicon tends to use single characters, while vernacular words often use pairs of characters. Take an excerpt from the Mencius as an example (Figure 1). The Mencius is a classic of Confucianism composed in classical Chinese. In the example, “ ” (being hungry) is used in isolation, but in vernacular is expected to be collocated with “ ” (“ ”) to express the same meaning. Second, classical has more frequent part-of-speech ambiguity. “ ” (clothes) is a noun in vernacular most of the time, but it functions as a verb when used before “ ” (silk), meaning “wear.” Third, word order in the classical register is more variable. In most cases, Chinese uses subject-verb-object order. In the classical clause “ ”, “ ” is the object and appears before the linking verb “ ”. This order is unconventional in vernacular Chinese.
Frequently, especially during the Ming and Qing periods, the boundary between vernacular and classical Chinese is not clear. Many texts mix the two registers in various ways. For example, dialog in classical texts often resembles the vernacular equivalent. Vernacular fiction also has a tradition of opening and closing a chapter with classical verse. The boundary blurred further when classical grammar was mixed with the vernacular lexicon at the end of the imperial period. In addition, written Chinese shares the same convention of having no obvious break markers (corresponding to punctuation) between “sentences” and delimiting paragraphs with empty spaces after the ending of a previous paragraph.1
In this study, we consider the task of cross-register authorship attribution using texts of known authorship. We aim to infer the authorship of a vernacular text given classical texts by the same author. Developing reliable cross-register authorship attribution techniques will be required to resolve the long-standing debates about disputed authorship of vernacular fictions, such as the Golden Plum Vase and the Marriage Destinies to Awaken the World, and roughly four hundred works as we found in our survey.
We assemble a corpus of eight authors known to have written in both registers. All authors lived between 1570 and 1870. All but one are from southern China, and all authors are men.2 The imbalance of gender and region reflects relevant social and economic circumstances of the period.3
The corpus contains 4.2 million characters of fictional and non-fictional prose, although these genres are not always distinct.4 Topics are diverse, including jokes, the care of pregnant women, history, opera commentary, war diaries, and personal reflections. All works address a general audience.
In practice, only authors who have written in each register and whose surviving works have machine-readable editions are considered.5 We refrain from picking texts that are disputed or use rhyme. We also avoid texts which are revisions of pre-existing texts or mixtures of vernacular glosses with classical grammar. Table 1 shows the texts in the corpus.
After downloading all texts, we performed the following preprocessing: 1. We checked all texts for flaws and missing parts by consulting other digital editions and print editions.6 If a character in a text lacks a Unicode code point, we used the modern 1Readers familiar with the evolution of Latin may gain some appreciation of how the registers difered by considering the lexical and syntactic diferences between Classical Latin (75 BCE to 300 CE) and Modern Latin (ca. 1500 - 1900 CE). The analogy is not exact, of course.
2We spent 20 hours searching for woman authors to include, but we were unable to find an author with
available texts. We gathered the candidate authors by consulting two bibliographies of Ming-Qing novels[
3At the time, education for women received less attention. And, southern China, such as Zhejiang and Jiangsu, was relatively wealthier and developed.
4Distinguishing between fictional and non-fictional historical narratives is difficult or, in some cases, impossible.
5The existence of an edition in a machine-readable format usually indicates a canonical author. This introduces a bias towards authors who were well known at the time or who subsequently became well known. 6If multiple versions exist, we choose the one which has fewer characters missing or obvious errors. variant. We refer to zdic.net and Unicodepedia to check whether a character falls outside of the Chinese Unicode set (between \u4e00 and \u9ff). We keep outside characters if they are valid Chinese characters or punctuation, deleting them otherwise.7 Characters are rarely deleted. 2. We remove title, heading, table of contents, preface, postscript, and editorial comments, as well as rhymed verse and prose if they are not part of the main body. Blank lines, redundant spaces, and indentation marks are removed too. 3. All the texts are automatically converted into UTF-8 encoded simplified Chinese using the Python package “hanziconv” (v.0.3.2). 4. Texts are then segmented into roughly 1,000-character chunks without breaking clauselevel structures. Modern publishers punctuate ancient Chinese texts, which originally did not have punctuation. We leverage these delimiters introduced by editors to avoid breaking clauses when segmenting. 5. We eliminate all punctuation in the next step to restore the original formatting. 6. For authors who have only one work in a register, we split the work into two parts.8 After performing these steps, we organize all chunks by register under each candidate’s directory with informative file names.
The corpus consists entirely of texts in the public domain and is available at https://zenodo. org/record/5513043.
Given our research question—Is it possible to recover the authorship of a vernacular text using
classical texts as training data?— we choose function words/characters for features because
they have been shown to be useful stylistic markers [
A linear support vector machine (SVM) is chosen as the classifier. We use LIBSVM’s
implementation [
8We do this to prevent severe inflation in calculating same-register accuracy. See justification for this treatment in the Appendix.
9Chinese vernacular function words overlap heavily with modern Chinese’s. Also, there is no function word
dictionary built for vernacular Chinese specifically, to the best of our knowledge. The modern function word
list [
10We released a Python package (“functionwords”) on PyPI to help others use these lists.
11A pilot study shows a standard logistic regression with L2 regularization (regularization parameter 1.0)
achieves similar accuracy as SVM [
Register classical classical vernacular vernacular vernacular vernacular classical vernacular classical classical vernacular classical vernacular vernacular vernacular classical vernacular classical classical vernacular classical vernacular
Genre consider authorship attribution in a scenario when a reasonable amount of classical training text is available. In the second experiment, we consider a scenario in which extensive classical training material is available.
We do not have a golden rule for how many classical Chinese characters constitute a large
amount of training material. For English language authorship attribution, Rao, Rohatgi, et al.
[
The experimental procedure is described using the limited data scenario. For a given candidate size from two to eight, we randomly choose four classical chunks from each author to ift the classifier. With the same trained classifier, we make predictions on test samples from diferent registers. The first prediction is made on ca. 1,000 vernacular characters, where the samples are randomly chosen from each author’s vernacular writing. We care about the classical-vernacular task the most because it shows how well a classifier trained with classical texts can successfully infer vernacular texts’ authorship.
The other prediction, the classical-classical task, uses 1,000 classical characters from each author. The classical-classical task’s testing samples are chosen from documents or parts of documents that are not in the training set to avoid inflating the accuracy (see the Appendix for a discussion of this concern). This task works as a baseline by indicating the level of accuracy the same classifier can perform with “vanilla” authorship attribution with classical Chinese. We also use random chance— candida1te size ×100%—as another baseline. The accuracy of the cross-register task should be bounded from above by the classical-classical accuracy and bounded from below by chance. The experiment is performed 2,000 times for every candidate size.
We predict the authorship of 1,000-character vernacular texts and 1,000-character classical Chinese texts with three feature sets under the limited data and abundant data scenarios. The classical-vernacular and classical-classical experiments use a standard linear SVM trained on the same training texts but evaluated on texts written in a diferent register (see Figure 2).
The classical-vernacular accuracy barely deviates from chance in the limited data scenario. Classical-vernacular accuracy is very slightly better than chance when trained on 10,000 character texts. In contrast, the classical-classical task, trained on the same data, performs far better than chance. With the optimal setting (combined feature set and 10,000 characters training data), the mean accuracy for classical-vernacular, classical-classical, and chance are 25.3%, 83.4%, and 24.5%, in turn.
The function characters/words feature sets play a trivial role in determining the cross-register accuracy but afect the classical-classical accuracy.
Confusion Matrix Confusion matrices show a similar pattern. For brevity, we only draw the confusion matrices under the abundant data scenario with the combined feature set assigning authorship given eight candidates (see Figure 3). The confusion matrices are normalized by rows (true labels).
In the confusion matrices, each row represents the true author, and each column represents the predicted author. Taking Feng Menglong (FML) as an example, the first row (labeled by “FML”) indicates the empirical probability of the linear SVM predicting the author indicated by the bottom labels when the true author is FML. In the classical-classical task (the left panel), FML has a probability of 0.6 to be predicted correctly; Du Gang (DG), Ding Yaokang (DYK), Wu Jingzi (WJZ), Chu Renhuo (CRH), and Li Yu (LY) have probabilities of 0.2, 0.1, 0.08, 0.05, and 0.006 to be misclassified as FML, respectively. However, CRH, DG, and LY are more likely to be predicted to be FML than the true author FML when it comes to a cross-register situation (the right panel).
In the classical-classical task, the values on diagonal are higher than random guess ( 18 = 0.125). Rarely is the classifier confused by authors with a similar writing style in the classical register. The most difficult case is predicting WJZ’s classical texts. Though CRH is a competitive candidate with a probability of 0.3 being assigned, the probability of correct prediction (0.4) is highest.
In the right panel, the probability of correct classification (the diagonal values) are much lower for classical-vernacular authorship attribution. Instead, CRH tends to be predicted as the author, regardless of the true author. DG, LY, and FML also attract incorrect attributions. Other authors receive little attention from the classifier.
A follow-up experiment Since CRH and FML are among favorite candidates in the crossregister task and both are known for their vernacular works, we investigate whether vernacularization plays a part in the main experiment. We made a follow-up experiment by removing the most favorite author from the candidate pool one by one. The queue follows the popularity (CRH, DG, LY, and FML, in turn).
The result shows that the next favored author keeps taking the lead by removing the most preferred. For instance, after removing the most popular author (CRH), the second popular author (DG) draws almost all the attention of the classifier when seven candidates are present.
Can we successfully infer the authorship of an unsigned vernacular text from classical texts with a standard authorship attribution technique? Our finding shows that inferring the author of a vernacular text using classical text as training data is challenging based on function characters/words frequency. Increasing training sample size is of limited help.
We speculate that the difficulty of the cross-register task lies in the elusive degree of “vernacularization” in classical texts. It is entirely possible that one’s classical style is “more vernacular” than others because the transition of Chinese from classical to vernacular unfolded gradually over time. For example, Chu Renhuo (CRH), the author most likely to be predicted by the model in the cross-register settings, is well-known as a vernacular novelist. In his classical writing, the Hard Gourd Collection, CRH documents many anecdotes about the composition of doggerel verse and rhymed verse (Qu) with extensive use of the function character “ ” (i.e., “ ” and “ ”).12 And “ ” as a modal particle rarely appears in written classical Chinese, if at all.
Also, the amount of dialog—another likely source of confusion—also varies across works. Dialog parts make a classical text resemble a vernacular text. For instance, Du Gang’s the Romance of the Northern Dynasties contains a large portion of dialog. This factor also relates to genre. Fiction and diaries tend to have more speech-like prose relative to history and poetry. The “signal” ordinarily picked up on by function words/characters is no longer detectable given this variability.
Future work might experiment with new, purpose-built feature sets which can mitigate the problem created by dialog and vernacularization. A larger corpus, especially one with limited dialog in classical texts, would also be valuable.
We thank the anonymous reviewers for their insightful comments on the manuscript of our 12Qu is one of the most colloquial form among rhymed verse forms in Chinese classical literature. paper. [16] E. Stamatatos. “Authorship attribution using text distortion”. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. 2017, pp. 1138–1149. [17] E. Stamatatos. “Masking topic-related information to enhance authorship attribution”.
In: Journal of the Association for Information Science and technology 69.3 (2018), pp. 461–473.
When processing the corpus, we performed a “create-two-from-one” strategy on the classical works for authors who only have one work. We did that because training and testing with texts from the same work can inflate the accuracy by overfitting a model to pick up topical information. Although function words are nominally content-free, genre and content can influence function word rates. Indeed, this problem is perhaps clearer in Chinese than it is in authorship attribution work using other languages (where the problem also exists). For example, (“very”) is a common classical function character, and it is part of the name of a Buddha, Mañjuśrī ( ). This may increase the likelihood of assigning a novel in which Mañjuśrī appears to an author who frequently uses as a function character.
To probe if our strategy prevents accuracy inflation, we ran another experiment only applying candidates who have at least two works in the classical register and compare it with the main experiment’s same-register accuracy (already with the “create-two-from-one” strategy used). Only three authors have at least two classical works. We calculated the same-register accuracy with diferent feature sets under the same setting corresponding to the limited data scenario of the main experiment (See Figure 4). By comparing the three-author settings, in one case strictly using diferent works and in the other case applying “create-two-from-one” strategy, we estimate that the accuracy inflation is low, less than 3%. Same-register accuracy is far better than chance.