This article solves the problem of the text authorship determining by analyzing the sentences structure. It is important to note that the task of the texts authorship establishing, as well as the task of its attribution, is still relevant for today and covers a wide range of goals in various fields and is interesting to a number of specialists in various fields.
To determine the true author of a text, it is often necessary to turn to experts who can identify the author of an unknown text or determine whether a work belongs to another author using characteristic linguistic features and various stylistic devices. Expert text analysis takes a lot of time and is very laborious. In this regard, formal methods of different texts attribution have great prospects for automating the analysis process.
Currently, various approaches such as the theory of pattern recognition, mathematical statistics and probability theory, algorithms of neural networks and cluster analysis, and many others are used for text attribution [1,2]. However, all methods used are not sufficiently effective. The particular difficulty is working with features that are characteristic of a particular language, which significantly complicates the task.
Working with the Ukrainian language, like other Slavic languages, has particular difficulties due to their structural complexity, as well as the variability of word forms and the possibilities of constructing sentences. Also, the complexity of the task is added by various styles of speech that are characteristic of a certain sphere of human activity, place of residence, age, education and subject of the text [3].
In this work, only literary works are used to determine authorship. The analysis of sentences in the text is carried out in order to form and formalize their structure.
Syntactic analysis of various text parts is a popular method for analyzing the author's work, its semantics, focus and main idea of the work. However, this type of various directions texts analysis is faced with the complexity of the syntactic model's automatic formation [4,5,6]. This is largely due to the complexity of the language structure itself, the variability of the word forms used and the very sentences structure. Despite this, method of text research like this carries the greatest amount of information about the author's style: regardless of the text subject, the syntactic structure of the author's language will clearly display his syllable.
Various studies of natural language formalization are known [7,8]. One of the methods to work with natural language is using of grammars [9]. For example, similar studies were carried out for Italian [10] and Ukrainian [3].
Unlike the problem of text categorization, the goal of which is to determine a topic or list of topics for a text based on its content, text parsing abstracts from a specific area and tries to understand contentindependent features of a text that are "linguistic expressions" of individual authors [11]. Such contentindependent text properties are usually called stylometric features. For these purposes, various methods have been proposed and applied in problems of authorship attribution, including: frequency of words [12,13], symbolic n-grams [14,15,16], auxiliary words, syllables [17] and parts of speech definition [18,19].
The idea of using information about parts of speech is not new and has been successfully applied in a number of style classification problems, where, in particular, texts in English were processed [19,2]. As a rule, their repeating sequences were extracted on the parts of speech basis.
The described approach is feasible for English texts, since the structure of the English language is quite strict and the word order in a sentence is clearly assigned to a certain part of speech. In addition, the words themselves do not have many different word forms, and when a prefix or suffix is added or removed, they go into another part of speech category. However, with using such method into Ukrainian, difficulties may arise. Unlike English, Ukrainian is a more variable language, and the number of forms for one word due to case, gender, and number changing significantly complicates the task. Moreover, the assignment of a word to one or another part of speech is ambiguous and difficult to perform within the automatic process.
The problem associated with establishing authorship need an individual approach in each case [20]. As a general rule, problems where the number of potential authors is small and the data samples are large are considered easy and high accuracy is expected. Complexity increases with an increase in the number of authors and a decrease in data volumes [21], which leads to a decrease in recognition accuracy.
This paper explores a method for the texts authorship determining based on the sentence structure of the author's individual language.
Stochastic grammar is used to create rules that describe the structure of sentences in a text. For each rule, the probability of its application in a particular work is determined. The probability of inferring the whole sentence is defined as the product of the parts of speech sequences probabilities used in it. The resulting rules will generate a language that is specific for the explored and structurally similar works of a certain author.
To describe the main text structure, the parts of speech as a word characteristic were used. Thus, each word in the sentence is replaced by the part of speech that it is.
Each of the words in the text was analyzed for similarity with the parts of speech existing in the Ukrainian language. For service parts of speech: prepositions, pronouns, conjunctions and interjections, their list was used in all possible forms, and verbs, nouns, adverbs, adjectives, participles and participles were determined by comparison with the list of word endings.
When a corresponding word or its ending was found in one of the lists, the word was automatically replaced with the corresponding part of speech. If it was not possible to automatically determine the answer, the user was asked to then include the entire word or its ending (the data was entered manually by the user) in an already existing list.
The following tags were used to tag words in the text in Ukrainian: verb (v), noun (n), pronoun (prn), adjective (adj), conjunction (cnj), adverb (adv), preposition (prp), participle (prtcpl), interjection (intrj), gerund (ger).
For each part of speech, the probability of its occurrence in a certain place of the sentence in the given text is calculated. The probability of the certain part of speech appearance in the studied sequence will allow us to more accurately capture the individual writing style specific of each of the authors under study. After receiving the text in the form of parts of speech sequences set in sentences with the probability of their occurrence in a particular place, rules are formed.
To do this, all sentences starting with the same part of speech are grouped, the first word is discarded, and the procedure for calculating the probability is repeated for the next word.
After the sentences are again grouped according to the parts of speech at the beginning, the first word is again discarded and the probability for the next element is calculated, and so on. Probability is calculated as the number of cases in the text divided by their total number. Thus, the substitution rules for some product T have an initial non-terminal, then terminals corresponding to each word in the sentence and the probability of applying the corresponding rule when parsing the text and have the form:
where σinitial non-terminal, ij bterminals corresponding to the i-th word in the sentence (and corresponding to the i-th rule applied when parsing the sentence or the i-th level of the rule), 𝐴 𝑖,𝑗j-th non-terminal in the i-th level rule, pi,kthe probability of applying the corresponding rule when parsing this work, Ji, Ktis the number of different non-terminals in the right part of the rules of the i-1-th level and i-th level, respectively.
The level corresponds to the ordinal number of the word in the sentence. Several alternative rules are allowed with a non-terminal on the left side of the rule, but the terminals on the right side of such rules are different, which ensures deterministic parsing. Thus, the text is presented as a set of rules that describe its structural features using the rules described above. The symbol ε stands for empty (end of rule).
An example of the one automatically restored set of rules is presented below in Table 1. The rule describes all 24 sentences in the text "Etude" by I. Bahrianyi, with a verb in the beginning. As can be seen from the presented probabilities, in the studied work, 31% of sentences will begin with a verb. And the percentage of sentences consisting of only one word, a verb, is 17%.
Examples of the first few rules according to the table are:
→ 𝑛𝐴 2,1 . On the left side of the rule is a non-terminal, then the probability of its application is indicated, and on the right side of the rule is a terminal with a non-terminal to go to the next rule.
When using this text method, a sentence from the work "Etude" by I. Bahrianyi presented as a sequence of parts of speech included in it will have the following form Table 2.
To compare two works, they must be presented in the form of a restored formal stochastic grammar with the rules, the formation of which is described above. Each sequence of rules in one text is compared with each sequence of rules in another text. Let the rules for some text Ti be formed like:
Let's say in texts Assume that two texts under study contain sentences of a similar structure ( i S and ' k S ), then the degree of their statistical structural similarity will be determined as the product of the minimum difference between the probabilities of applying the corresponding rule: Formation of formal stochastic grammars was carried out for all the works of each author in the training sample, generating the language specific for a particular author. For determining the similarity of a work according to (1), the formal stochastic grammar corresponding to the work from the control sample was used as T1, and the stochastic grammar for all works of the potential author altogether was used as T2.
To obtain more reliable results, we calculated confidence intervals for each of the authors in the sample. Student's t-test was applied [22].
To calculate the confidence interval in the training sample for each of the authors, their presented texts similarities to each other were calculated. Data on the similarity of texts within the training sample for each of the authors was divided into three parts with the same number of components.
The following formula was used to calculate the confidence interval:
, where t2,β -Student's t-test, βconfidence level, ζkthe average value of k-th sample part, θSthe average value over the entire sample.
During the experiment, the authorship of natural language texts was determined by two samples.
For the first experiment, 20 works of literary texts by 10 Ukrainian authors were selected in the training sample. The control sample consists of 3 works by each author.
The works of the following authors are presented: IB -I. Bahrianyi, AV -A. Vyshnia, MV -M. Vovchok, AD -A. Dovzhenko, HK -H. Kvitka-Osnovianenko, PM -P. Myrnyi, VN -V. Nestaiko, VP -V. Pidmohylnyi, IF -I. Franko, MK -M. Khvylovyi.
For the second experiment, both samples were doubled, respectively, the training sample included 40 works by the same authors, and 60 texts made up a new control sample -6 works by each author.
The choice of literary texts is due to the availability of reliable information about the works authorship and the presence of each author specific style.
At Figure 1 and Figure 2 the results of the experiments are presented. Each bar in the chart represents the works of a particular author from the control sample. The column is divided into two zones, where the blue part displays the number of texts with correctly identified authorship, and the orange part shows the number of texts with erroneous ones.
According to the results obtained, working with the smaller of the two samples, containing a total of 200 works by 10 authors in the training sample (20 for each author), and 30 works in the control sample, cases of authorship correct attribution is 24, which is 80%.
The best result was obtained working with the works of A. Dovzhenko, P. Myrnyi, V. Nestaiko and V. Pidmohylnyi. I. Franko turned out to be the author with the most difficult to define style.
According to theobtained results, working with a larger sample (400 texts of 10 authors in the training sample and 60 works in the control sample), the number of correctly determined cases is 45, which is 75%.
The best result was obtained working with the works of P. Myrny. The authors of the most difficult cases to detect are H. Kvitka-Osnovianenko and M. Vovchok -the athurship of only a half of their works in the sample was correctly determined.
To obtain better result, a confidence interval was introduced Table 3. Confidence intervals of different authors differ significantly from each other. So, on average, the interval ranges from 0.04 to 0.08, however, for A. Vyshnia and M. Vovchok it is much larger, and amounted to 0.37 and 0.12, respectively. The minimum interval was 0.02 for V. Pidmohylnyi.
For some of the authors, such as H. Kvitka-Osnovianenko and M. Vovchok, a special style of narration is characteristic, which is difficult to classify and structure, due to which they are characterized by a low recognition result.
True False
Whereas the authors I. Bahrianyi, A. Vyshnia, A. Dovzhenko, P. Myrnbi and V. Nestaiko have a more individual style of writing, which is displayed in the sentence structure and allows establishing their authorship with high accuracy.
AV MV AD HK PM VN VP IF MKH Average 0,83 0,83 0,61 0,83 0,58 0,97 0,87 0,69 0,67 0,66 Max 0,86 1,00 0,67 0,86 0,60 0,99 0,89 0,70 0,71 0,69 Min 0,80 0,64 0,55 0,80 0,56 0,95 0,85 0,68 0,63 0,63 Range 0,05 0,37 0,12 0,06 0,05 0,04 0,03 0,02 0,07 0,07
Taking into account the confidence intervals, the following results were obtained Table 4.
Results of authorship determination working with the confidence interval In the interval 1 actual author According to the results obtained, for 14 cases, as a result of authorship determination, one correctly recognized candidate was obtained, which amounted to 23.3%. In 36 cases, the program was able to narrow the number of applicants to 3, while in 31 cases the applicant with the greatest structural similarity was the correctly recognized author (which accounted for 51.7% of the total sample) and in another 5 cases the correctly recognized author was included in the list of candidates (8.3% of the total sample). Also, in only 8 cases, the authorship of the text was not determined -none of the submitted candidates was correctly recognized, which amounted to 13.3%.
As a result of the obtained data analysis, it can be argued that the size of the confidence interval can also be considered a specific feature of the author's personal style. So, for some of them, the size of the confidence interval differs significantly from other authors. It is much larger in A. Vyshnia and M. Vovchok. It is noteworthy that A. Vyshnia is characterized by his own style of writing, which made it possible to determine with sufficient accuracy the texts of his authorship, while the style of M. Vovchok is rather difficult to classify. The minimum interval was obtained for the works of the author V. Pidmohylnyi, which, however, leads to rather high results of his classification.
Thus, the percentage of correct authorship identification, taking into account the confidence interval, was improved to 83.33% (in 50 cases out of 60, the author of the work was correctly recognized).
In a study of English poets works authorship [30] using the architectures of a convolutional neural network, a multilayer perceptron, and LSTM neural network, the results ranged from 74-83%. In another work, using various stylometric functions and algorithms, also in English authors works, the success rate was 82% [20]. Analyzing text corpora in English and Spanish, working with syllables, the achieved result was 78.8%. For the Russian language, which is similar in complexity and structure to Ukrainian, working with a combination of support vector machine and a genetic algorithm, such results as 82.3% were achieved [27]. And using N-grams in processing the Portuguese language in [31], the result reached 72%.
Working with Ukrainian texts, predominantly journalistic style, in view of the structural variability and complexity of the language, using methods such as neural networks [28] and the Quantitative Method for Automated Text Authorship Attribution Based on the Statistical Analysis of N-grams Distribution [29] and working with scientific articles, the authors obtained results of 92% and 79%, respectively.
The case of using confidence intervals in determining the authorship of texts has not been found in recent works, which allows us to assert the novelty of this approach.
Among the works related to text tagging, one can find works devoted to the N-grams [6,33,34].
For example, a study [6] of the Coptic language, which is the last phase of the Egyptian language family and a descendant of the ancient Egyptian script, was conducted to assess the success of tagging the study of genre, style and authorship in the Coptic language. The results of the study show a relatively high accuracy of 94-95% correct automatic tagging for literary texts.
In [33] the authors focused on the attribution of the Polish texts authorship using stylometric features based on part-of-speech tags. The Polish language is characterized by a high level of inflection, so the authors managed to distinguish more than 1000 tags, which made it possible to build a fairly large feature space by processing texts and performing their classification using machine learning methods. The use of this method in highly inflected languages, including Polish, is considered by the authors to be a promising direction in authorship attribution.
In [34] for the authorship attribution problem, the use of part-of-speech skip-grams and an in-house top-k sequential pattern mining algorithm is considered. The authors of the study come to an accuracy of 86-97% for various authors in training sample.
Given the differences in the analyzed languages in the presented studies, we can conclude that the method proposed here is a promising direction for working with Ukrainian literary texts and will significantly improve the results obtained.
The paper proposes a new method of text attribution using stochastic formal grammars. Since all known methods do not give high accuracy and do not take into account the sentence constructing rules, there is a need to search for new additional methods and new attributes. The characteristic of the author's style in the aspect of the sentence constructing is a previously unexplored sphere. Its use, in combination with already known methods, can increase the efficiency of determining the natural language texts authorship.
Analyzing texts with a sample 20x3, the number of matches for the author was 24 out of 30 works. And working with doubled samples, the result is also positive, but to a lesser extent -45 out of 60 matches. The results obtained were 80% and 75%, respectively.
Taking into account the confidence interval, the results were improved to 83.3%. As a result of the analysis, it can be argued that the size of the confidence interval can also be considered a characteristic feature of the author's personal style. Thus, a large confidence interval may indicate a low level of differentiation of the author's style and, as a result, a poor result in determining the authorship of his works. And vice versa -with a small confidence interval, the probability of confident the author's style differentiation increases significantly.
The average value of the works similarity in the training sample is also significant -the higher the value, the more clearly the style of the author is determined and, accordingly, the result of determining his works authorship is higher.