Natural Language Texts Authorship Establishing was carried out on the basis of the hypothesis that each author has a peculiar the sentences structure forming style with different parts of speech. A natural language text was translated into a formal language generated by a formal stochastic grammar. For each product of the training sample, the corresponding stochastic formal grammar was restored. This method made it possible to reflect the author characteristic style in sentences building. On the basis of works statistical sample, inference rules and a probabilistic measure of their application were formed. The effectiveness of the proposed method was evaluated experimentally. In authorship establishing a probabilistic measure of the text belonging to a formal stochastic grammar was determined. To assess the reliability of the obtained results, the confidence interval of the probability measure was calculated. In the studies with the control sample, the possibility of the correct text authorship establishment is natural language texts, statistic analysis, text structure, text authorship, classification, parsing, confidence interval, formal stochastic grammar This article solves the problem of the text authorship determining by analyzing the sentences structure. It is important to note that the task of the texts authorship establishing, as well as the task of its attribution, is still relevant for today and covers a wide range of goals in various fields and is interesting to a number of specialists in various fields.
2022 Copyright for this paper by its authors.
Syntactic analysis of various text parts is a popular method for analyzing the author's work, its
semantics, focus and main idea of the work. However, this type of various directions texts analysis is
faced with the complexity of the syntactic model’s automatic formation [
Various studies of natural language formalization are known [
Unlike the problem of text categorization, the goal of which is to determine a topic or list of topics
for a text based on its content, text parsing abstracts from a specific area and tries to understand
contentindependent features of a text that are "linguistic expressions" of individual authors [
The idea of using information about parts of speech is not new and has been successfully applied in
a number of style classification problems, where, in particular, texts in English were processed [
The described approach is feasible for English texts, since the structure of the English language is quite strict and the word order in a sentence is clearly assigned to a certain part of speech. In addition, the words themselves do not have many different word forms, and when a prefix or suffix is added or removed, they go into another part of speech category. However, with using such method into Ukrainian, difficulties may arise. Unlike English, Ukrainian is a more variable language, and the number of forms for one word due to case, gender, and number changing significantly complicates the task. Moreover, the assignment of a word to one or another part of speech is ambiguous and difficult to perform within the automatic process.
The problem associated with establishing authorship need an individual approach in each case [20]. As a general rule, problems where the number of potential authors is small and the data samples are large are considered easy and high accuracy is expected. Complexity increases with an increase in the number of authors and a decrease in data volumes [21], which leads to a decrease in recognition accuracy.
This paper explores a method for the texts authorship determining based on the sentence structure of the author's individual language.
Stochastic grammar is used to create rules that describe the structure of sentences in a text. For each rule, the probability of its application in a particular work is determined. The probability of inferring the whole sentence is defined as the product of the parts of speech sequences probabilities used in it. The resulting rules will generate a language that is specific for the explored and structurally similar works of a certain author.
To describe the main text structure, the parts of speech as a word characteristic were used. Thus, each word in the sentence is replaced by the part of speech that it is.
Each of the words in the text was analyzed for similarity with the parts of speech existing in the Ukrainian language. For service parts of speech: prepositions, pronouns, conjunctions and interjections, their list was used in all possible forms, and verbs, nouns, adverbs, adjectives, participles and participles were determined by comparison with the list of word endings.
When a corresponding word or its ending was found in one of the lists, the word was automatically replaced with the corresponding part of speech. If it was not possible to automatically determine the answer, the user was asked to then include the entire word or its ending (the data was entered manually by the user) in an already existing list.
The following tags were used to tag words in the text in Ukrainian: verb (v), noun (n), pronoun (prn), adjective (adj), conjunction (cnj), adverb (adv), preposition (prp), participle (prtcpl), interjection (intrj), gerund (ger).
For each part of speech, the probability of its occurrence in a certain place of the sentence in the given text is calculated. The probability of the certain part of speech appearance in the studied sequence will allow us to more accurately capture the individual writing style specific of each of the authors under study. After receiving the text in the form of parts of speech sequences set in sentences with the probability of their occurrence in a particular place, rules are formed.
To do this, all sentences starting with the same part of speech are grouped, the first word is discarded, and the procedure for calculating the probability is repeated for the next word.
After the sentences are again grouped according to the parts of speech at the beginning, the first word is again discarded and the probability for the next element is calculated, and so on. Probability is calculated as the number of cases in the text divided by their total number. corresponding to the i-th rule applied when parsing the sentence or the i-th level of the rule), , – j-th non-terminal in the i-th level rule, pi,k – the probability of applying the corresponding rule when parsing this work, Ji, Kt – is the number of different non-terminals in the right part of the rules of the i-1-th level and i-th level, respectively.
The level corresponds to the ordinal number of the word in the sentence. Several alternative rules are allowed with a non-terminal on the left side of the rule, but the terminals on the right side of such rules are different, which ensures deterministic parsing. Thus, the text is presented as a set of rules that describe its structural features using the rules described above. The symbol ε stands for empty (end of rule).
An example of the one automatically restored set of rules is presented below in Table 1. The rule describes all 24 sentences in the text "Etude" by I. Bahrianyi, with a verb in the beginning. As can be seen from the presented probabilities, in the studied work, 31% of sentences will begin with a verb. And the percentage of sentences consisting of only one word, a verb, is 17%.
Examples of the first few rules according to the table 1,1; 0,17 0,13 1,1 → ; 1,1 → 2,1 . On the left side of the rule is a non-terminal, then the probability of its application is indicated, and on the right side of the rule is a terminal with a non-terminal to go to the next rule.
When using this text method, a sentence from the work "Etude" by I. Bahrianyi presented as a sequence of parts of speech included in it will have the following form Table 2.
To compare two works, they must be presented in the form of a restored formal stochastic grammar with the rules, the formation of which is described above. Each sequence of rules in one text is compared with each sequence of rules in another text. Let the rules for some text Ti be formed like: ' →p'1j b'1 j A'1, j , A'i, j p→'i+1,k b'i+1,k A'i+1,k , j = 1 J 'i , k = 1 K 'i .
Let's say in texts Ti and Tk there are sentences of similar structure, such as pljl bljl in any of these works, if the text T2 structurally similar sentence to sentence Si is absent, then (Si , Si' ) = 0 .
notice, that (Ti ,Tk ) = (Tk ,Ti ) (Ti ,Ti ) = 1 complete match, (Ti ,Tk ) = 0 – if in texts Ti and Tk no sentences have the same structure.
Formation of formal stochastic grammars was carried out for all the works of each author in the training sample, generating the language specific for a particular author. For determining the similarity of a work according to (1), the formal stochastic grammar corresponding to the work from the control sample was used as T1, and the stochastic grammar for all works of the potential author altogether was used as T2.
To obtain more reliable results, we calculated confidence intervals for each of the authors in the sample. Student's t-test was applied [22].
To calculate the confidence interval in the training sample for each of the authors, their presented texts similarities to each other were calculated. Data on the similarity of texts within the training sample for each of the authors was divided into three parts with the same number of components.
The following formula was used to calculate the confidence interval:
2, √61 ∑3=1(ζ − )2, where t2,β – Student's t-test, β – confidence level, ζk – the average value of k-th sample part, θS – the average value over the entire sample.
During the experiment, the authorship of natural language texts was determined by two samples.
For the first experiment, 20 works of literary texts by 10 Ukrainian authors were selected in the training sample. The control sample consists of 3 works by each author.
The works of the following authors are presented: IB – I. Bahrianyi, AV – A. Vyshnia, MV – M. Vovchok, AD – A. Dovzhenko, HK – H. Kvitka-Osnovianenko, PM – P. Myrnyi, VN – V. Nestaiko, VP – V. Pidmohylnyi, IF – I. Franko, MK – M. Khvylovyi.
For the second experiment, both samples were doubled, respectively, the training sample included 40 works by the same authors, and 60 texts made up a new control sample - 6 works by each author.
The choice of literary texts is due to the availability of reliable information about the works authorship and the presence of each author specific style.
At Figure 1 and Figure 2 the results of the experiments are presented. Each bar in the chart represents the works of a particular author from the control sample. The column is divided into two zones, where the blue part displays the number of texts with correctly identified authorship, and the orange part shows the number of texts with erroneous ones. IB
AV
MV
AD
HK True
PM False
VN
VP
IF
MKH
According to the results obtained, working with the smaller of the two samples, containing a total of 200 works by 10 authors in the training sample (20 for each author), and 30 works in the control sample, cases of authorship correct attribution is 24, which is 80%.
The best result was obtained working with the works of A. Dovzhenko, P. Myrnyi, V. Nestaiko and V. Pidmohylnyi. I. Franko turned out to be the author with the most difficult to define style.
Sample 40x6 IB
AV
MV
AD
VN
VP
IF
MKH HK True
PM False
According to theobtained results, working with a larger doubled sample (400 texts of 10 authors in the training sample and 60 works in the control sample), the number of correctly determined cases is 45, which is 75%.
The best result was obtained working with the works of P. Myrny. The authors of the most difficult cases to detect are H. Kvitka-Osnovianenko and M. Vovchok - the athurship of only a half of their works in the sample was correctly determined.
To obtain better result, a confidence interval was introduced Table 3.
Confidence intervals of different authors differ significantly from each other. So, on average, the interval ranges from 0.04 to 0.08, however, for A. Vyshnia and M. Vovchok it is much larger, and amounted to 0.37 and 0.12, respectively. The minimum interval was 0.02 for V. Pidmohylnyi.
For some of the authors, such as H. Kvitka-Osnovianenko and M. Vovchok, a special style of narration is characteristic, which is difficult to classify and structure, due to which they are characterized by a low recognition result.
Whereas the authors I. Bahrianyi, A. Vyshnia, A. Dovzhenko, P. Myrnbi and V. Nestaiko have a more individual style of writing, which is displayed in the sentence structure and allows establishing their authorship with high accuracy.
According to the results obtained, for 14 cases, as a result of authorship determination, one correctly recognized candidate was obtained, which amounted to 23.3%. In 36 cases, the program was able to narrow the number of applicants to 3, while in 31 cases the applicant with the greatest structural similarity was the correctly recognized author (which accounted for 51.7% of the total sample) and in another 5 cases the correctly recognized author was included in the list of candidates (8.3% of the total sample). Also, in only 8 cases, the authorship of the text was not determined - none of the submitted candidates was correctly recognized, which amounted to 13.3%.
As a result of the obtained data analysis, it can be argued that the size of the confidence interval can also be considered a specific feature of the author's personal style. So, for some of them, the size of the confidence interval differs significantly from other authors. It is much larger in A. Vyshnia and M. Vovchok. It is noteworthy that A. Vyshnia is characterized by his own style of writing, which made it possible to determine with sufficient accuracy the texts of his authorship, while the style of M. Vovchok is rather difficult to classify. The minimum interval was obtained for the works of the author V. Pidmohylnyi, which, however, leads to rather high results of his classification.
Thus, the percentage of correct authorship identification, taking into account the confidence interval, was improved to 83.33% (in 50 cases out of 60, the author of the work was correctly recognized).
According to our previous studies [
In this paper, the proposed method was used for the first time and results of 75-80% were obtained, which, taking into account the peculiarities of the Ukrainian language and the complexity of its formalization for solving the task, correspond to the spread in the percentage of determining the texts authorship in works devoted to this topic. So, for example, the result of determining authorship is in the range from 74% to 92% correctly identified cases [26-32]. These results varied depending on the used method, the language and style of the analyzed text.
Working with different foreign languages and taking into account their distinctive features, the authors used various methods.
In a study of English poets works authorship [30] using the architectures of a convolutional neural network, a multilayer perceptron, and LSTM neural network, the results ranged from 74-83%. In another work, using various stylometric functions and algorithms, also in English authors works, the success rate was 82% [20]. Analyzing text corpora in English and Spanish, working with syllables, the achieved result was 78.8%. For the Russian language, which is similar in complexity and structure to Ukrainian, working with a combination of support vector machine and a genetic algorithm, such results as 82.3% were achieved [27]. And using N-grams in processing the Portuguese language in [31], the result reached 72%.
Working with Ukrainian texts, predominantly journalistic style, in view of the structural variability and complexity of the language, using methods such as neural networks [28] and the Quantitative Method for Automated Text Authorship Attribution Based on the Statistical Analysis of N-grams Distribution [29] and working with scientific articles, the authors obtained results of 92% and 79%, respectively.
The case of using confidence intervals in determining the authorship of texts has not been found in recent works, which allows us to assert the novelty of this approach.
Among the works related to text tagging, one can find works devoted to the N-grams [
For example, a study [
In [33] the authors focused on the attribution of the Polish texts authorship using stylometric features based on part-of-speech tags. The Polish language is characterized by a high level of inflection, so the authors managed to distinguish more than 1000 tags, which made it possible to build a fairly large feature space by processing texts and performing their classification using machine learning methods. The use of this method in highly inflected languages, including Polish, is considered by the authors to be a promising direction in authorship attribution.
In [34] for the authorship attribution problem, the use of part-of-speech skip-grams and an in-house top-k sequential pattern mining algorithm is considered. The authors of the study come to an accuracy of 86-97% for various authors in training sample.
Given the differences in the analyzed languages in the presented studies, we can conclude that the method proposed here is a promising direction for working with Ukrainian literary texts and will significantly improve the results obtained.
The paper proposes a new method of text attribution using stochastic formal grammars. Since all known methods do not give high accuracy and do not take into account the sentence constructing rules, there is a need to search for new additional methods and new attributes. The characteristic of the author’s style in the aspect of the sentence constructing is a previously unexplored sphere. Its use, in combination with already known methods, can increase the efficiency of determining the natural language texts authorship.
Analyzing texts with a sample 20x3, the number of matches for the author was 24 out of 30 works. And working with doubled samples, the result is also positive, but to a lesser extent - 45 out of 60 matches. The results obtained were 80% and 75%, respectively.
Taking into account the confidence interval, the results were improved to 83.3%. As a result of the analysis, it can be argued that the size of the confidence interval can also be considered a characteristic feature of the author's personal style. Thus, a large confidence interval may indicate a low level of differentiation of the author's style and, as a result, a poor result in determining the authorship of his works. And vice versa - with a small confidence interval, the probability of confident the author's style differentiation increases significantly.
The average value of the works similarity in the training sample is also significant - the higher the value, the more clearly the style of the author is determined and, accordingly, the result of determining his works authorship is higher.
In the future, it is planned to improve the presented method to obtain a better result, and for the same purpose, it can be combined with previously studied methods. The possibility of more detailed tagging of sentences parts is considered - working not only parts of speech, but also forms, numbers, gender, etc. for the word under study. This approach will provide more information about the structure of sentences and the rules for their construction, specific to a particular author.
[17] G. O. Sidorov, Automatic Authorship Attribution Using Syllables as Classification Features,
Rhema journal 1 (2018) 62-81. [18] E. Stamatatos, A survey of modern authorship attribution methods, J. Am. Soc. Inf. Sci. Technol 60 (3) (2009) 538–556. [19] M. Koppel, J. Schler, S. Argamon, Computational methods in authorship attribution, J. Am. Soc.
Inf. Sci. Technol 60 (1) (2009) 9–26. doi:10.1002/asi.20961. [20] M. Koppel, J. Schler, S. Argamon, Authorship attribution: what’s easy and what’s hard?, Lecture
Notes in Computer Science 7499 (2012) 282-289. doi:10.1007/978-3-642-32790-2_34. [21] K. Luyckx, W. Daelemans, The effect of author set size and data size in authorship attribution, Lit.
Linguist. Comput. 26 (1) (2011) 35–55. [22] P. Mishra, U. Singh, C. M. Pandey, P. Mishra, G. Pandey, Application of student's t-test, analysis of variance, and covariance, Annals of cardiac anaesthesia 22(4) (2019) 407–411. doi:10.4103/aca.ACA_94_19. [23] V. I. Shynkarenko, I. M. Demidovich, Determination of the attributes of authorship of natural texts,
Artificial Intelligence 3 (2018) 27-35. [24] V. I. Shynkarenko, I. M. Demidovich Authorship Determination of Natural Language Texts by Several Classes of Indicators with Customizable Weights, in: Proceedings of the 5th International Conference on Computational Linguistics and Intelligent Systems (COLINS 2021), Volume I: Main Conference, Lviv, Ukraine, April 22-23, 2021, pp. 832-844. [25] I. Demidovich, V. Shynkarenko, O. Kuropiatnyk, O. Kirichenko, Processing Words Effectiveness Analysis in Solving the Natural Language Texts Authorship Determination Task, in: Proceedings of the XVI International Scientific and Technical Conference (CSIT'2021), Lviv, Ukraine, 2021. doi: 10.1109/CSIT52700.2021.9648829. [26] R. Iyer, C. Rosé, A Machine Learning Framework for Authorship Identification FromTexts. ArXiv abs/1912.10204 (2019). [27] J. Rygl, A. Horák, Authorship Attribution: Comparison of Single-Layer and Double-Layer Machine Learning, Lecture Notes in Computer Science 7499 (2012) 282-289. doi:10.1007/9783-642-32790-2_34. [28] M. Lupei, A. Mitsa, V. Repariuk, V. Sharkan, Identification of authorship of Ukrainian-language texts of journalistic style using neural networks, Eastern-European Journal of Enterprise Technologies 1(2) (2020) 30–36. doi:10.15587/1729-4061.2020.195041. [29] V. Lytvyn et al., Development of the Quantitative Method for Automated Text Content Authorship Attribution Based on the Statistical Analysis of N-grams Distribution, Eastern-European Journal of Enterprise Technologies 6(2) (2019) 28-51. doi:10.15587/1729-4061.2019.186834. [30] V. Moshkina, I. Andreeva, N. Yarushkinaa, Solving the problem of determining the author of text data using a combined assessment, CEUR Workshop 2782 (2020) 112-118. [31] I. Markov, J. Baptista, O. Pichardo-Lagunas, Authorship Attribution in Portuguese Using Character N-grams, Acta Polytechnica Hungarica 14(3) (2017) 59–78. doi:10.12700/APH.14.3.2017.3.4. [32] G. O. Sidorov, Automatic Authorship Attribution Using Syllables as Classification Features,
Rhema journal 1 (2018) 62-81. [33] P. Szwed Authorship Attribution for Polish Texts Based on Part of Speech Tagging. In: Kozielski S., Mrozek D., Kasprowski P., Małysiak-Mrozek B., Kostrzewa D. (eds) Beyond Databases, Architectures and Structures. Towards Efficient Solutions for Data Analysis and Knowledge Representation. BDAS 2017. Communications in Computer and Information Science, vol 716.
Springer, Cham. (2017) doi:10.1007/978-3-319-58274-0_26. [34] J. Pokou, F. Fournier-Viger, Ch. Moghrabi, Using Frequent Fixed or Variable-Length POS Ngrams or Skip-Grams for Blog Authorship Attribution, CPCI-S 收录 (2019) 63-74.