This article discusses the problem of searching for non-uniform text fragments. Each text fragment consists of several paragraphs or separate sentences which significantly difer from the rest of the text in terms of a set of characteristics. The problem of finding non-uniform fragments and their interpretation arises in the study of the pre-revolutionary magazines ”Time” (1861-1863), ”Epoch” (1864-1865) and the weekly ”Citizen” (1873-1874). It's a known fact that F. M. Dostoevsky was their editor. It means that he could have made his own edits to the texts of articles written by other authors. In our research the texts were divided into separate parts. For every part the frequency of n-grams (encoded sequences of parts of speech) was determined. Further, the analysis was carried out using decision trees that classified texts by author. In particular, the texts of F. M. Dostoevsky and V. P. Meshchersky were subjected to this analysis.
A fragment of text that consists of several paragraphs or even separate sentences
will be called non-uniform if it difers significantly from the rest of the text in
terms of a set of characteristics. Each characteristic should be poorly controlled
by the author of the work and be able to statistically separate two or more
authors. For example, for this purpose (to distinguish the text of F. M. Dostoevsky,
A. Grigoriev, V. Dahl), the frequency of diferent sequences of parts of speech
found in fragments was used [
At the same time, it is important to select the most informative features that
allow you to identify non-uniform fragments [
Based on the corpus of prosaic texts by Russian writers of the XVIII-XX
centuries (215 texts by 50 authors) and scientific articles on philology, history, law,
economics and other social and humanitarian sciences (500 texts written without
co-authorship), the authors conclude that the method is quite accurate. Units
of the symbolic level of the text, elements of grammar, idiosyncratic and special
features of the text were used as characteristics to be compared, including [
A close task is the identification of artificial texts [
A significant disadvantage of the considered algorithms is the poor interpretability of the results, which is very important when solving the problem of attribution. In addition, modern neural classifiers require a large amount of training sample for their construction. Decision trees and forests difer better in this case.
The work consists of four sections and a conclusion. The introduction describes the problem of detecting non-uniform fragments in the text and the existing solutions. The second section shows how, based on the frequency of occurrence of certain n-grams, decision trees are constructed for the problem of attribution of texts from the magazines ”Time” (1861-1863), ”Epoch” (1864-1865) and the weekly ”Citizen” (1873-1874). The third part provides a mathematical model used for attribution of a text fragment. The fourth section describes the developed tools for visualizing research results (highlighting text fragments, highlighting n-grams, coloring text), implemented in the SMALT information system. 2
Description of the Method Based on the Decision Tree The problem of finding non-uniform fragments and their interpretation arises in the study of the pre-revolutionary magazines ”Time” (1861-1863), ”Epoch” (1864-1865) and the weekly ”Citizen” (1873-1874). It’s a known fact that the famous Russian writer F. M. Dostoevsky was their editor. It means that he could have made his own edits to the texts of articles of other authors. There is a number of works that have been published without the author’s name (or under a pseudonym), which also allows specialists in the field of literary studies to formulate attributive hypotheses.
Since any hypothesis need a convincing proof, mathematical methods that
complement philological research are becoming more and more popular.
Previously, the authors have already used decision tree method and it has shown
good results. For example, in [
Note that the rfist work with the use of decision trees appeared in the 60s of
the 20th century and since then they are often used by data mining specialists.
There are Classification trees and Regression trees [
In order to carry out such an analysis, it is necessary to split the source
texts into fragments, while setting the left and right borders. Then grammatical
markup is carried out, which takes into account 14 parts of speech (noun,
adjective, numeral, pronoun, adverb, category of state, verb, participle, gerund,
preposition, conjunction, particle, modal word, interjection) and also allows to mark
quotes, foreign words, introductory words, abbreviated words and non-linguistic
symbols. Sequences of parts of speech of diferent lengths can be represented
as n-grams (n indicates how many elements should be taken and determines
the size of the sequence). There are bigrams (2-grams), trigrams (3-grams),
4grams, 5-grams, etc. When we talk about unigrams, or in other words n-grams
consisting of one element, we mean the words themselves. N-grams are widely
used in natural language processing: for example, for prompting the next word
in a search string, searching for plagiarism, correcting errors, etc. Let’s take a
look at a well-known example of determining the authorship of the novel ”The
Quiet Don”. Some critics argued that M. Sholokhov was too young at the time
of writing the novel (23 years old), while the novel was very mature. There
was a hypothesis that the author was Fyodor Kryukov, who also wrote about
the life of cossacks. To test this hypothesis the frequency of the combination of
diferent classes of words at the level of bigrams, trigrams and tetragrams was
studied [
The studies carried out by the authors have shown that bigrams are suitable
for solving the problems of attribution of texts from the journals ”Time”
(18611863), ”Epoch” (1864-1865) and the weekly ”Citizen” (1873-1874) [
After that the text No. 160 ”Dvoryanin, zhelayushchij byt’ krest’yaninom” (”A nobleman who wants to be a peasant”) was analyzed, which was published in the magazine ”Time” (1861, volume VI, No. 12, pp. 117-123). However, it is dificult for specialists in philology to analyze the tree in this representation. In order to simplify this process, an algorithm for attributing text fragments was developed. 3
The Attribution Model of a Text Fragment The method of attribution of a text fragment proposed in the report is based on the ensemble of classifiers. Let’s select the fragment in the text that needs to be attributed, denote it by x. With each fragment x we will associate a binary variable z, which will take the value 0 if the text belongs to one author and value 1 if it belongs to the other. Let this fragment x is contained in j fragments yj whose classification is known. Let’s denote it by vj . The general classification of all fragments is denoted by f (v1, ..., vj ).
As an example, you can use: j f (v1, ..., vj ) = 1 X vk j
k=1 f (v1, ..., vj ) = (1, if Pjk=1 vk ≥ 2j 0, otherwise
If f (v1, ..., vj ) is a binary function, then the obvious solution would be z = f (v1, ..., vj ). If we have not one tree, but a forest of decisions, then the function f (v1, ..., vj ) may have a diferent form.
When it is required to take into account a possible rejection of the classification (for example, when j is an even number), then the variable zi must take three values. To the values 0 and 1, 1/2 must be added, as a rejection of the classification
Then the formula (2) will take the form: 11,/2, iiff PPjjkk==11 vvkk >= 22jj 0, otherwise (3)
Note that this technique can be generalized to the case of evaluating the degree of belonging of a given fragment to the author (high, medium, low and excluded) or evaluating the probability of belonging.
Let’s consider an example of the proposed algorithm based on the constructed decision tree. In our case, the size of the fragment x is equal to the split step. As a criterion we used the formula (3). The results of the algorithm are presented in Table 1. Here, the first author is F. M. Dostoevsky and the second author is V. P. Meshchersky. The text in question is marked with a number 160 ”Dvoryanin, zhelayushchij byt’ krest’yaninom” (”A nobleman who wants to be a peasant”). The unit of measurement is the word. Based on the source data, the SMALT system builds a text that is colored in accordance with the identity of the author (this is described in more detail in the next section).
The authors applied this approach when a transformer model was used as a classifier. The results are presented on the resource http://smalt.karelia.ru/. A significant limitation of the use of the classifier built on the transformer model is the need for a large volume of the training sample.
Due to the absence of marked texts with foreign fragments, it is not possible to assess the accuracy of the proposed method. However, philological experts rated it highly. Their expert assessment of some fragments coincided with the assessment obtained using the method described in this report. 4
Software Support for Text Analysis Methods and
Algorithms
A large number of routine operations is required to identify non-uniform
fragments: marking up texts, calculating statistics and analyzing the results. Usually
such tasks are performed by a team of specialists, where the problem of
interaction and exchange of the obtained results arises. The information system
”Statistical methods of literary text analysis” (SMALT) allows you to speed up these
operations at the expense of computer technologies. SMALT has a modular
structure [
Information system SMALT allows you to get the marked-up text in the form of a table in the format xls (Microsoft Excel), csv or ods (Librofice Calc). The table contains three columns: – source word; – initial form; – part of speech code.
Sentences are separated by an empty line, paragraphs are separated by two empty lines. If there is loaded text with punctuation marks, a punctuation mark is displayed instead of an empty line in the first column. The results of text processing are submitted in JSON format as an array of objects containing the following fields: – id : the sequence number of the fragment; – left border : the number of the first word (numbering from zero); – right border : the number of the last word; – pros: assessment of the degree of belonging to the first author; – cons: assessment of the degree of belonging to the second author.
To visualize the obtained results, SMALT provides the following tools: – selection of a fragment of text; – highlighting n-gram; – coloring text.
Displaying fragments of text allows you to select the required fragment in a text work. To view the text fragments go to the view form through the ”Research” → ”Text Fragments” menu. The form consists of fields for defining the selection parameters (fragment size, fragment number, indent size) and a block for displaying the selection results. To select a fragment its size (sample size) and also its ofset relative to the beginning of the text are used. The ofset is calculated based on the indent number and its size (for example 3 indent of size 10 gives an ofset of 30 words). The indent number is the fragment number.
The system allows you to specify several fragments using the generally accepted notation of enumeration and range. For example, for a fragment size of 15 and an indent size of 10, specifying fragment numbers in the form ”1-3,5,7” results in the selection of the following word ranges: from 10 to 45, from 50 to 65, and from 70 to 85. The highlighting of the n-gram is done in the same form. To do this in the fields ”Part No. 1”, ”Part No. 2” and ”Part No. 3” you need to select the required part of speech (see Fig. 1). You can also show only matches at the beginning of sentences by specifying the appropriate flag. You can search for unigram, bigram and trigram.
Coloring text allows you to visually highlight text fragments. The selection is performed in accordance with the formula (3). There are three options for coloring: – yellow: f (v1, ..., vj ) = 1; – green: f (v1, ..., vj ) = 1/2; – white: f (v1, ..., vj ) = 0.
Insertions of quotations in the text do not participate in the coloring of the text, they are marked in italics. Text coloring pages are stored in the information system database. To download the coloring book you must specify the text, the description of the experiment and the json-file with the distribution of votes by fragments in the appropriate form. This article depicts the problem of searching and interpreting non-uniform text fragments based on the material of the pre-revolutionary journals ”Time” (18611863), ”Epoch” (1864-1865) and the weekly ”Citizen” (1873-1874). The model of attribution of text fragments based on a heuristic algorithm using decision trees has been developed. The used sings were the frequency of the occurrence of certain n-grams (encoded sequences of parts of the speech). To analyze the obtained results in the SMALT information system (http://smalt.karelia.ru/), tools for highlighting text fragments, highlighting n-grams and coloring text have been implemented.
Acknowledgements. This work was supported by the Russian Foundation for Basic Research, project no. 18-012-90026.