Natural Language Texts Authorship Establishing Based on the Sentences Structure Viktor Shynkarenko and Inna Demidovich Ukrainian State University of Science and Technologies, 2, аcademician Lazaryan str., Dnipro, 49010, Ukraine Abstract Natural Language Texts Authorship Establishing was carried out on the basis of the hypothesis that each author has a peculiar the sentences structure forming style with different parts of speech. A natural language text was translated into a formal language generated by a formal stochastic grammar. For each product of the training sample, the corresponding stochastic formal grammar was restored. This method made it possible to reflect the author characteristic style in sentences building. On the basis of works statistical sample, inference rules and a probabilistic measure of their application were formed. The effectiveness of the proposed method was evaluated experimentally. In authorship establishing a probabilistic measure of the text belonging to a formal stochastic grammar was determined. To assess the reliability of the obtained results, the confidence interval of the probability measure was calculated. In the studies with the control sample, the possibility of the correct text authorship establishment is 75-80%. Keywords natural language texts, statistic analysis, text structure, text authorship, classification, parsing, confidence interval, formal stochastic grammar 1. Introduction This article solves the problem of the text authorship determining by analyzing the sentences structure. It is important to note that the task of the texts authorship establishing, as well as the task of its attribution, is still relevant for today and covers a wide range of goals in various fields and is interesting to a number of specialists in various fields. To determine the true author of a text, it is often necessary to turn to experts who can identify the author of an unknown text or determine whether a work belongs to another author using characteristic linguistic features and various stylistic devices. Expert text analysis takes a lot of time and is very laborious. In this regard, formal methods of different texts attribution have great prospects for automating the analysis process. Currently, various approaches such as the theory of pattern recognition, mathematical statistics and probability theory, algorithms of neural networks and cluster analysis, and many others are used for text attribution [1, 2]. However, all methods used are not sufficiently effective. The particular difficulty is working with features that are characteristic of a particular language, which significantly complicates the task. Working with the Ukrainian language, like other Slavic languages, has particular difficulties due to their structural complexity, as well as the variability of word forms and the possibilities of constructing sentences. Also, the complexity of the task is added by various styles of speech that are characteristic of a certain sphere of human activity, place of residence, age, education and subject of the text [3]. In this work, only literary works are used to determine authorship. The analysis of sentences in the text is carried out in order to form and formalize their structure. COLINS-2022: 6th International Conference on Computational Linguistics and Intelligent Systems, May 12–13, 2022, Gliwice, Poland EMAIL: shinkarenko_vi@ua.fm (V. Shynkarenko); 2019demidovichinn@gmail.com (I. Demidovich) ORCID: 0000-0001-8738-7225 (V. Shynkarenko); 0000-0002-3644-184X (I. Demidovich) ©️ 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 2. Related works Syntactic analysis of various text parts is a popular method for analyzing the author's work, its semantics, focus and main idea of the work. However, this type of various directions texts analysis is faced with the complexity of the syntactic model’s automatic formation [4, 5, 6]. This is largely due to the complexity of the language structure itself, the variability of the word forms used and the very sentences structure. Despite this, method of text research like this carries the greatest amount of information about the author's style: regardless of the text subject, the syntactic structure of the author's language will clearly display his syllable. Various studies of natural language formalization are known [7, 8]. One of the methods to work with natural language is using of grammars [9]. For example, similar studies were carried out for Italian [10] and Ukrainian [3]. Unlike the problem of text categorization, the goal of which is to determine a topic or list of topics for a text based on its content, text parsing abstracts from a specific area and tries to understand content- independent features of a text that are "linguistic expressions" of individual authors [11]. Such content- independent text properties are usually called stylometric features. For these purposes, various methods have been proposed and applied in problems of authorship attribution, including: frequency of words [12, 13], symbolic n-grams [14, 15, 16], auxiliary words, syllables [17] and parts of speech definition [18, 19]. The idea of using information about parts of speech is not new and has been successfully applied in a number of style classification problems, where, in particular, texts in English were processed [19, 2]. As a rule, their repeating sequences were extracted on the parts of speech basis. The described approach is feasible for English texts, since the structure of the English language is quite strict and the word order in a sentence is clearly assigned to a certain part of speech. In addition, the words themselves do not have many different word forms, and when a prefix or suffix is added or removed, they go into another part of speech category. However, with using such method into Ukrainian, difficulties may arise. Unlike English, Ukrainian is a more variable language, and the number of forms for one word due to case, gender, and number changing significantly complicates the task. Moreover, the assignment of a word to one or another part of speech is ambiguous and difficult to perform within the automatic process. The problem associated with establishing authorship need an individual approach in each case [20]. As a general rule, problems where the number of potential authors is small and the data samples are large are considered easy and high accuracy is expected. Complexity increases with an increase in the number of authors and a decrease in data volumes [21], which leads to a decrease in recognition accuracy. 3. Methods 3.1. Work structure rules formation This paper explores a method for the texts authorship determining based on the sentence structure of the author's individual language. Stochastic grammar is used to create rules that describe the structure of sentences in a text. For each rule, the probability of its application in a particular work is determined. The probability of inferring the whole sentence is defined as the product of the parts of speech sequences probabilities used in it. The resulting rules will generate a language that is specific for the explored and structurally similar works of a certain author. To describe the main text structure, the parts of speech as a word characteristic were used. Thus, each word in the sentence is replaced by the part of speech that it is. Each of the words in the text was analyzed for similarity with the parts of speech existing in the Ukrainian language. For service parts of speech: prepositions, pronouns, conjunctions and interjections, their list was used in all possible forms, and verbs, nouns, adverbs, adjectives, participles and participles were determined by comparison with the list of word endings. When a corresponding word or its ending was found in one of the lists, the word was automatically replaced with the corresponding part of speech. If it was not possible to automatically determine the answer, the user was asked to then include the entire word or its ending (the data was entered manually by the user) in an already existing list. The following tags were used to tag words in the text in Ukrainian: verb (v), noun (n), pronoun (prn), adjective (adj), conjunction (cnj), adverb (adv), preposition (prp), participle (prtcpl), interjection (intrj), gerund (ger). For each part of speech, the probability of its occurrence in a certain place of the sentence in the given text is calculated. The probability of the certain part of speech appearance in the studied sequence will allow us to more accurately capture the individual writing style specific of each of the authors under study. After receiving the text in the form of parts of speech sequences set in sentences with the probability of their occurrence in a particular place, rules are formed. To do this, all sentences starting with the same part of speech are grouped, the first word is discarded, and the procedure for calculating the probability is repeated for the next word. After the sentences are again grouped according to the parts of speech at the beginning, the first word is again discarded and the probability for the next element is calculated, and so on. Probability is calculated as the number of cases in the text divided by their total number. Table 1 The rules of the restored stochastic grammar according to "Etude" by I. Bahrianyi Left Left Probability Terminal Nonterminal Probability Terminal Nonterminal part part σ 0,31 v A1,1 A3,4 1,00 prp A4,3 A1,1 0,17 ε A3,5 1,00 adv A3,3 A1,1 0,13 n A2,1 A3,6 1,00 v A4,4 A1,1 0,04 prp A2,2 A3,7 0,80 adj A3,7 A1,1 0,04 cnj A2,3 A3,7 0,20 ε A1,1 0,21 v A2,4 A4,1 1,00 intrj A5,1 A1,1 0,13 prn A2,5 A4,2 1,00 adj+n G3 A2,1 0,40 ε A4,3 1,00 n A5,2 A2,1 0,40 n A3,1 A4,4 1,00 adv A5,3 A2,1 0,20 prp A3,2 A4,5 1,00 v A5,4 A2,2 1,00 n A3,3 A5,1 1,00 n A6,1 A2,3 1,00 v A3,1 A5,2 0,33 ε A2,4 0,20 adj+n A4,5 A5,2 0,33 v A6,2 A2,4 0,20 ε A5,2 0,33 adv A6,1 A2,4 0,20 prp A2,1 A5,3 1,00 n A3,3 A2,4 0,20 v A3,4 A5,4 1,00 prp A6,3 A2,4 0,20 prn A3,5 A6,1 1,00 v A7,1 A2,5 0,33 n A3,6 A6,2 1,00 v A7,3 A2,5 0,33 adj A3,7 A6,3 1,00 prtcpl A7,2 A2,5 0,33 v A3,4 A7,1 1,00 prp A5,3 A3,1 0,50 ε A7,2 0,33 adj+n A7,2 A3,1 0,20 cnj A4,1 A7,2 0,67 ε A3,2 1,00 prn A4,2 A7,3 1,00 n A3,3 A3,3 1,00 ε Thus, the substitution rules for some product T have an initial non-terminal, then terminals corresponding to each word in the sentence and the probability of applying the corresponding rule when parsing the text and have the form: p1 j pi +1,k  → b1 j A1, j , Ai , j → bi +1, k Ai +1, k , j = 1 Ji , k = 1 Ki where σ – initial non-terminal, bij – terminals corresponding to the i-th word in the sentence (and corresponding to the i-th rule applied when parsing the sentence or the i-th level of the rule), 𝐴𝑖,𝑗 – j-th non-terminal in the i-th level rule, pi,k – the probability of applying the corresponding rule when parsing this work, Ji, Kt – is the number of different non-terminals in the right part of the rules of the i-1-th level and i-th level, respectively. The level corresponds to the ordinal number of the word in the sentence. Several alternative rules are allowed with a non-terminal on the left side of the rule, but the terminals on the right side of such rules are different, which ensures deterministic parsing. Thus, the text is presented as a set of rules that describe its structural features using the rules described above. The symbol ε stands for empty (end of rule). An example of the one automatically restored set of rules is presented below in Table 1. The rule describes all 24 sentences in the text "Etude" by I. Bahrianyi, with a verb in the beginning. As can be seen from the presented probabilities, in the studied work, 31% of sentences will begin with a verb. And the percentage of sentences consisting of only one word, a verb, is 17%. 0.31 Examples of the first few rules according to the table are: 𝜎→ 𝑣𝐴1,1 ; 0,17 0,13 𝐴1,1 → 𝜀; 𝐴1,1 → 𝑛𝐴2,1 . On the left side of the rule is a non-terminal, then the probability of its application is indicated, and on the right side of the rule is a terminal with a non-terminal to go to the next rule. When using this text method, a sentence from the work "Etude" by I. Bahrianyi presented as a sequence of parts of speech included in it will have the following form Table 2. Table 2 Sentence tagging and corresponding probabilities of stochastic grammar rules Word in a sentence Tag Probability Чорні adj 0,06 ґрати n 0,6 розпанахали v 0,6 небо n 0,125 3.2. Comparison of two works To compare two works, they must be presented in the form of a restored formal stochastic grammar with the rules, the formation of which is described above. Each sequence of rules in one text is compared with each sequence of rules in another text. Let the rules for some text Ti be formed like: p'1 j p'i +1,k  → b A , A i , j → b'i +1, k A'i +1, k , j = 1 ' ' 1j ' 1, j ' J 'i , k = 1 K 'i . Let's say in texts Ti and Tk there are sentences of similar structure, such as p1 j1 p2 j2 pljl Si :  b1 j1 A1 j1  b2 j2 A2 j2  bljl p'1 j1 p'2 j2 p'ljl Sk :  b'1 j1 A'1 j1  b'2 j2 A'2 j2  b'ljl и biji = b 'iji Assume that two texts under study contain sentences of a similar structure ( Si and S 'k ), then the degree of their statistical structural similarity will be determined as the product of the minimum difference between the probabilities of applying the corresponding rule: l  ( Si , S 'k ) =  min( pmj − pmj i m )m m =1 And the degree of two works texts statistical structural similarity as the sum of the all its sentences similarity degrees. The degree of texts statistical structural similarity Ti and Tk : N  (T1 , T2 ) =   ( Si , S 'i ), (1) i =1 where S 'i – structurally similar sentence to sentence Si according (1), N – the number of sentences in any of these works, if the text T2 structurally similar sentence to sentence Si is absent, then  ( Si , Si' ) = 0 . notice, that  (Ti , Tk ) =  (Tk , Ti )  (Ti , Ti ) = 1 complete match,  (Ti , Tk ) = 0 – if in texts Ti and Tk no sentences have the same structure. Formation of formal stochastic grammars was carried out for all the works of each author in the training sample, generating the language specific for a particular author. For determining the similarity of a work according to (1), the formal stochastic grammar corresponding to the work from the control sample was used as T1, and the stochastic grammar for all works of the potential author altogether was used as T2. 3.3. Calculation of the confidence interval boundary values using Student's t-test To obtain more reliable results, we calculated confidence intervals for each of the authors in the sample. Student's t-test was applied [22]. To calculate the confidence interval in the training sample for each of the authors, their presented texts similarities to each other were calculated. Data on the similarity of texts within the training sample for each of the authors was divided into three parts with the same number of components. The following formula was used to calculate the confidence interval: 1 𝑡2,𝛽 √6 ∑3𝑘=1(ζ𝑘 − 𝜃𝑠 )2 , where t2,β – Student's t-test, β – confidence level, ζk – the average value of k-th sample part, θS – the average value over the entire sample. 4. Results 4.1. Training and control samples formation During the experiment, the authorship of natural language texts was determined by two samples. For the first experiment, 20 works of literary texts by 10 Ukrainian authors were selected in the training sample. The control sample consists of 3 works by each author. The works of the following authors are presented: IB – I. Bahrianyi, AV – A. Vyshnia, MV – M. Vovchok, AD – A. Dovzhenko, HK – H. Kvitka-Osnovianenko, PM – P. Myrnyi, VN – V. Nestaiko, VP – V. Pidmohylnyi, IF – I. Franko, MK – M. Khvylovyi. For the second experiment, both samples were doubled, respectively, the training sample included 40 works by the same authors, and 60 texts made up a new control sample - 6 works by each author. The choice of literary texts is due to the availability of reliable information about the works authorship and the presence of each author specific style. 4.2. Experiment results At Figure 1 and Figure 2 the results of the experiments are presented. Each bar in the chart represents the works of a particular author from the control sample. The column is divided into two zones, where the blue part displays the number of texts with correctly identified authorship, and the orange part shows the number of texts with erroneous ones. Sample 20x3 3 2 1 0 IB AV MV AD HK PM VN VP IF MKH True False Figure 1: Authorship establishing results with a sample 20x3 According to the results obtained, working with the smaller of the two samples, containing a total of 200 works by 10 authors in the training sample (20 for each author), and 30 works in the control sample, cases of authorship correct attribution is 24, which is 80%. The best result was obtained working with the works of A. Dovzhenko, P. Myrnyi, V. Nestaiko and V. Pidmohylnyi. I. Franko turned out to be the author with the most difficult to define style. Sample 40x6 6 5 4 3 2 1 0 IB AV MV AD HK PM VN VP IF MKH True False Figure 2: Authorship establishing results with a sample 40x6 According to theobtained results, working with a larger doubled sample (400 texts of 10 authors in the training sample and 60 works in the control sample), the number of correctly determined cases is 45, which is 75%. The best result was obtained working with the works of P. Myrny. The authors of the most difficult cases to detect are H. Kvitka-Osnovianenko and M. Vovchok - the athurship of only a half of their works in the sample was correctly determined. To obtain better result, a confidence interval was introduced Table 3. Confidence intervals of different authors differ significantly from each other. So, on average, the interval ranges from 0.04 to 0.08, however, for A. Vyshnia and M. Vovchok it is much larger, and amounted to 0.37 and 0.12, respectively. The minimum interval was 0.02 for V. Pidmohylnyi. For some of the authors, such as H. Kvitka-Osnovianenko and M. Vovchok, a special style of narration is characteristic, which is difficult to classify and structure, due to which they are characterized by a low recognition result. Whereas the authors I. Bahrianyi, A. Vyshnia, A. Dovzhenko, P. Myrnbi and V. Nestaiko have a more individual style of writing, which is displayed in the sentence structure and allows establishing their authorship with high accuracy. Table 3 Confidence interval by authors IB AV MV AD HK PM VN VP IF MKH Average 0,83 0,83 0,61 0,83 0,58 0,97 0,87 0,69 0,67 0,66 Max 0,86 1,00 0,67 0,86 0,60 0,99 0,89 0,70 0,71 0,69 Min 0,80 0,64 0,55 0,80 0,56 0,95 0,85 0,68 0,63 0,63 Range 0,05 0,37 0,12 0,06 0,05 0,04 0,03 0,02 0,07 0,07 Taking into account the confidence intervals, the following results were obtained Table 4. Table 4 Results of authorship determination working with the confidence interval In the interval 1 actual 1 not actual more than 1 author, more than 1 author, author author actual is among them actual isn’t among them Number 14 2 36 8 Per cent 23,3% 3,3% 60% 13,3% According to the results obtained, for 14 cases, as a result of authorship determination, one correctly recognized candidate was obtained, which amounted to 23.3%. In 36 cases, the program was able to narrow the number of applicants to 3, while in 31 cases the applicant with the greatest structural similarity was the correctly recognized author (which accounted for 51.7% of the total sample) and in another 5 cases the correctly recognized author was included in the list of candidates (8.3% of the total sample). Also, in only 8 cases, the authorship of the text was not determined - none of the submitted candidates was correctly recognized, which amounted to 13.3%. As a result of the obtained data analysis, it can be argued that the size of the confidence interval can also be considered a specific feature of the author's personal style. So, for some of them, the size of the confidence interval differs significantly from other authors. It is much larger in A. Vyshnia and M. Vovchok. It is noteworthy that A. Vyshnia is characterized by his own style of writing, which made it possible to determine with sufficient accuracy the texts of his authorship, while the style of M. Vovchok is rather difficult to classify. The minimum interval was obtained for the works of the author V. Pidmohylnyi, which, however, leads to rather high results of his classification. Thus, the percentage of correct authorship identification, taking into account the confidence interval, was improved to 83.33% (in 50 cases out of 60, the author of the work was correctly recognized). 5. Discussion According to our previous studies [3, 23, 24, 25] on establishing the texts authorship, the initial result of reliability ranged from 18% of the correct texts attribution to 82% [23] using character-by- character analysis. Later, the result was improved with a range from 80% to 91% using N-grams [24], genetic algorithm [25] and also working with stems and dictionaries [3]. In this paper, the proposed method was used for the first time and results of 75-80% were obtained, which, taking into account the peculiarities of the Ukrainian language and the complexity of its formalization for solving the task, correspond to the spread in the percentage of determining the texts authorship in works devoted to this topic. So, for example, the result of determining authorship is in the range from 74% to 92% correctly identified cases [26-32]. These results varied depending on the used method, the language and style of the analyzed text. Working with different foreign languages and taking into account their distinctive features, the authors used various methods. In a study of English poets works authorship [30] using the architectures of a convolutional neural network, a multilayer perceptron, and LSTM neural network, the results ranged from 74-83%. In another work, using various stylometric functions and algorithms, also in English authors works, the success rate was 82% [20]. Analyzing text corpora in English and Spanish, working with syllables, the achieved result was 78.8%. For the Russian language, which is similar in complexity and structure to Ukrainian, working with a combination of support vector machine and a genetic algorithm, such results as 82.3% were achieved [27]. And using N-grams in processing the Portuguese language in [31], the result reached 72%. Working with Ukrainian texts, predominantly journalistic style, in view of the structural variability and complexity of the language, using methods such as neural networks [28] and the Quantitative Method for Automated Text Authorship Attribution Based on the Statistical Analysis of N-grams Distribution [29] and working with scientific articles, the authors obtained results of 92% and 79%, respectively. The case of using confidence intervals in determining the authorship of texts has not been found in recent works, which allows us to assert the novelty of this approach. Among the works related to text tagging, one can find works devoted to the N-grams [6, 33, 34]. For example, a study [6] of the Coptic language, which is the last phase of the Egyptian language family and a descendant of the ancient Egyptian script, was conducted to assess the success of tagging the study of genre, style and authorship in the Coptic language. The results of the study show a relatively high accuracy of 94-95% correct automatic tagging for literary texts. In [33] the authors focused on the attribution of the Polish texts authorship using stylometric features based on part-of-speech tags. The Polish language is characterized by a high level of inflection, so the authors managed to distinguish more than 1000 tags, which made it possible to build a fairly large feature space by processing texts and performing their classification using machine learning methods. The use of this method in highly inflected languages, including Polish, is considered by the authors to be a promising direction in authorship attribution. In [34] for the authorship attribution problem, the use of part-of-speech skip-grams and an in-house top-k sequential pattern mining algorithm is considered. The authors of the study come to an accuracy of 86-97% for various authors in training sample. Given the differences in the analyzed languages in the presented studies, we can conclude that the method proposed here is a promising direction for working with Ukrainian literary texts and will significantly improve the results obtained. 6. Conclusions The paper proposes a new method of text attribution using stochastic formal grammars. Since all known methods do not give high accuracy and do not take into account the sentence constructing rules, there is a need to search for new additional methods and new attributes. The characteristic of the author’s style in the aspect of the sentence constructing is a previously unexplored sphere. Its use, in combination with already known methods, can increase the efficiency of determining the natural language texts authorship. Analyzing texts with a sample 20x3, the number of matches for the author was 24 out of 30 works. And working with doubled samples, the result is also positive, but to a lesser extent - 45 out of 60 matches. The results obtained were 80% and 75%, respectively. Taking into account the confidence interval, the results were improved to 83.3%. As a result of the analysis, it can be argued that the size of the confidence interval can also be considered a characteristic feature of the author's personal style. Thus, a large confidence interval may indicate a low level of differentiation of the author's style and, as a result, a poor result in determining the authorship of his works. And vice versa - with a small confidence interval, the probability of confident the author's style differentiation increases significantly. The average value of the works similarity in the training sample is also significant - the higher the value, the more clearly the style of the author is determined and, accordingly, the result of determining his works authorship is higher. In the future, it is planned to improve the presented method to obtain a better result, and for the same purpose, it can be combined with previously studied methods. The possibility of more detailed tagging of sentences parts is considered - working not only parts of speech, but also forms, numbers, gender, etc. for the word under study. This approach will provide more information about the structure of sentences and the rules for their construction, specific to a particular author. 7. References [1] R. A. Hardcastle, CUSUM: a credible method for the determination of authorship?, Science & Justice: Journal of the Forensic Science Society 37(2) (1997) 129-138. doi:10.1016/s1355- 0306(97)72158-0. [2] P Juola, GK Mikros, S Vinsick, A comparative assessment of the difficulty of authorship attribution in Greek and in English, Journal of the Association for Information Science and Technology 70 (1) (2019) 61-70. doi:10.1002/asi.24073. [3] V. Shynkarenko, O. Kuropiatnyk, Constructive Model of the Natural Language, Acta Cybernetica. 23, 4 (2018) 995–1015. doi:10.14232/actacyb.23.4.2018.2. [4] Y. Zhang, H. Kamigaito, M. Okumura, A Language Model-based Generative Classifier for Sentence-level Discourse Parsing, in: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Online and Punta Cana, Dominican Republic, Association for Computational Linguistics, 2021, pp. 2432–2446. doi:10.18653/v1/2021.emnlp-main.188. [5] J. Li, M. Liu, B. Qin, et al. A survey of discourse parsing, Front. Comput. Sci. 16 (2022). doi:10.1007/s11704-021-0500-z. [6] A. Zeldes, C. T. Schroeder, Computational Methods for Coptic: Developing and Using Part-of- Speech Tagging for Digital Scholarship in the Humanities, Digital Scholarship in the Humanities 30 (2015) i164–i176. doi:10.1093/llc/fqv043. [7] I. Buzhinsky, Formalization of natural language requirements into temporal logics: a survey, 2019 IEEE 17th INDIN, (2019), 400-406. doi:10.1109/INDIN41052.2019.8972130. [8] P. Linz, SH. Rodger, An introduction to formal languages and automata, Jones & Bartlett Learning, 2022. [9] M. Silberztein, A new linguistic engine for nooj: Parsing context-sensitive grammars with finite- state machines, in: Proceedings of the 11th International NooJ Conference Formalizing Natural Languages with NooJ and Its Natural Language Processing Applications, Kenitra, Morocco, 2017, pp. 240– 250. doi:10.1007/978-3-319-73420-0_20. [10] A. Mazzei, V. Lombardo, Building a large grammar for Italian, in: LREC, 2004. [11] M. Gamon, Linguistic correlates of style: authorship classification with deep linguistic analysis features, in: Proceedings of the 20th International Conference on Computational Linguistics. COLING 2004, Stroudsburg. Association for Computational Linguistics, 2004, doi:10.3115/1220355.1220443. [12] C. Akimushkin, DR. Amancio, ON. Oliveira Jr., On the role of words in the network structure of texts: Application to authorship attribution, Physica A: Statistical Mechanics and its Applications 495 (2018) 49-58. doi:10.1016/j.physa.2017.12.054. [13] DL. Hoover, The microanalysis of style variation, Digital Scholarship in the Humanities, Issue suppl_2 32 (2017) ii17-30. doi: 10.1093/llc/fqx022. [14] Y. Sari, A. Vlachos, M. Stevenson, Continuous N-gram Representations for Authorship Attribution, in: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, Valencia, Spain, 2017, pp. 267-273. doi:10.18653/v1/E17-2043. [15] I. Markov, J. Baptista, O. Pichardo-Lagunas, Authorship Attribution in Portuguese Using Character N-grams, Acta Polytechnica Hungarica 14(3) (2017) 59–78. doi:10.12700/APH.14.3.2017.3.4. [16] H. Gómez-Adorno, JP. Posadas-Durán, G. Sidorov, Document embeddings learned on various types of n-grams for cross-topic authorship attribution, Computing 100 (2018) 741–756. doi:10.1007/s00607-018-0587-8. [17] G. O. Sidorov, Automatic Authorship Attribution Using Syllables as Classification Features, Rhema journal 1 (2018) 62-81. [18] E. Stamatatos, A survey of modern authorship attribution methods, J. Am. Soc. Inf. Sci. Technol 60 (3) (2009) 538–556. [19] M. Koppel, J. Schler, S. Argamon, Computational methods in authorship attribution, J. Am. Soc. Inf. Sci. Technol 60 (1) (2009) 9–26. doi:10.1002/asi.20961. [20] M. Koppel, J. Schler, S. Argamon, Authorship attribution: what’s easy and what’s hard?, Lecture Notes in Computer Science 7499 (2012) 282-289. doi:10.1007/978-3-642-32790-2_34. [21] K. Luyckx, W. Daelemans, The effect of author set size and data size in authorship attribution, Lit. Linguist. Comput. 26 (1) (2011) 35–55. [22] P. Mishra, U. Singh, C. M. Pandey, P. Mishra, G. Pandey, Application of student's t-test, analysis of variance, and covariance, Annals of cardiac anaesthesia 22(4) (2019) 407–411. doi:10.4103/aca.ACA_94_19. [23] V. I. Shynkarenko, I. M. Demidovich, Determination of the attributes of authorship of natural texts, Artificial Intelligence 3 (2018) 27-35. [24] V. I. Shynkarenko, I. M. Demidovich Authorship Determination of Natural Language Texts by Several Classes of Indicators with Customizable Weights, in: Proceedings of the 5th International Conference on Computational Linguistics and Intelligent Systems (COLINS 2021), Volume I: Main Conference, Lviv, Ukraine, April 22-23, 2021, pp. 832-844. [25] I. Demidovich, V. Shynkarenko, O. Kuropiatnyk, O. Kirichenko, Processing Words Effectiveness Analysis in Solving the Natural Language Texts Authorship Determination Task, in: Proceedings of the XVI International Scientific and Technical Conference (CSIT'2021), Lviv, Ukraine, 2021. doi: 10.1109/CSIT52700.2021.9648829. [26] R. Iyer, C. Rosé, A Machine Learning Framework for Authorship Identification FromTexts. ArXiv abs/1912.10204 (2019). [27] J. Rygl, A. Horák, Authorship Attribution: Comparison of Single-Layer and Double-Layer Machine Learning, Lecture Notes in Computer Science 7499 (2012) 282-289. doi:10.1007/978- 3-642-32790-2_34. [28] M. Lupei, A. Mitsa, V. Repariuk, V. Sharkan, Identification of authorship of Ukrainian-language texts of journalistic style using neural networks, Eastern-European Journal of Enterprise Technologies 1(2) (2020) 30–36. doi:10.15587/1729-4061.2020.195041. [29] V. Lytvyn et al., Development of the Quantitative Method for Automated Text Content Authorship Attribution Based on the Statistical Analysis of N-grams Distribution, Eastern-European Journal of Enterprise Technologies 6(2) (2019) 28-51. doi:10.15587/1729-4061.2019.186834. [30] V. Moshkina, I. Andreeva, N. Yarushkinaa, Solving the problem of determining the author of text data using a combined assessment, CEUR Workshop 2782 (2020) 112-118. [31] I. Markov, J. Baptista, O. Pichardo-Lagunas, Authorship Attribution in Portuguese Using Character N-grams, Acta Polytechnica Hungarica 14(3) (2017) 59–78. doi:10.12700/APH.14.3.2017.3.4. [32] G. O. Sidorov, Automatic Authorship Attribution Using Syllables as Classification Features, Rhema journal 1 (2018) 62-81. [33] P. Szwed Authorship Attribution for Polish Texts Based on Part of Speech Tagging. In: Kozielski S., Mrozek D., Kasprowski P., Małysiak-Mrozek B., Kostrzewa D. (eds) Beyond Databases, Architectures and Structures. Towards Efficient Solutions for Data Analysis and Knowledge Representation. BDAS 2017. Communications in Computer and Information Science, vol 716. Springer, Cham. (2017) doi:10.1007/978-3-319-58274-0_26. [34] J. Pokou, F. Fournier-Viger, Ch. Moghrabi, Using Frequent Fixed or Variable-Length POS Ngrams or Skip-Grams for Blog Authorship Attribution, CPCI-S 收录 (2019) 63-74.