The novelty of the research is an offered combination of the machine learning method - the data clustering and the classical method - the Student's t-test to differentiate English and Ukrainian texts. The efficiency of the two methods has been proved to be high for determining the style factor effect and the authorial style factor effect. The research allows us to conclude that the data clustering is a simpler method than the Student's t-test, but it ensures essential differences in fewer cases than the Student's t-test. The use of the Student's t-test is more complicated as it can be performed only after the Pearson's normality test. However, with the help of the Student's t-test, the essential differences have been established in most cases with a test validity of 95%. The research shows that the proposed combination of methods ensures reliable results. The obtained results may be used for text analysis and authorship attribution. Data clustering, Student's t-test, Style factor effect, Authorial style factor effect, Authorship COLINS-2022: 6th International Conference on Computational Linguistics and Intelligent Systems, May 12-13, 2022, Gliwice, Poland ORCID: 0000-0003-3470-7191 (I. Khomytska); 0000-0002-5974-9310 (V. Teslyuk); 0000-0002-5391-0556 (Yu. Kordiika)
attribution
The problem raised in the research is closely connected with text analysis. Text differentiation
implies identifying the text distinctive features. There are different approaches to text analysis. They
can be classified according to the language level (phonological, lexical, syntactic) and language units.
All the approaches aim at characterizing specificity of the researched functional style or authorial style.
The machine learning methods are widely used for text analysis [
The quantitative approach is used for feminism studies in Ukraine [
2022 Copyright for this paper by its authors.
[
The purpose of our research is to determine an efficient combination of the machine learning and
the classical methods which ensures high test validity results for text differentiating. The novel approach
consists in offering a combination of the data clustering and the Student’s t-test for differentiating
English and Ukrainian texts. In our previous research, the Student’s t-test proved to be efficient on the
phonological level. The authors were differentiated by consonant phoneme groups [13-15]. This method
was also successfully applied on the lexical level [16]. The data clustering method is efficient on the
same language levels – the phonological and lexical levels [
The latest methodologies and approaches aim at an optimal solution of the problem of text differentiation. The solution must be simple and it must ensure high accuracy. The problem is not easy to solve as the authorial features are not often clear-cut. The degree of clarity of authorial style features must be sufficient. An author may use the vocabulary common for certain sphere of communication. Because of this, the authorial style lacks the distinctive individual features, by which the manner of writing of one author can be differentiated from that of another author. In fiction, the author’s writing is peculiar and can be easily characterized. In scientific papers and formal documents, the author’s manner of writing can hardly be noticeable. In this case, different approaches are used to define the differentiating features of this piece of writing. Therefore, we propose a combination of the machine learning and the classical methods. The data clustering ensures a simple solution of text differentiation. The Student’s t-test ensures reliable results.
The research is done on the lexical level (function words) and the phonological level (consonant phoneme groups) [18]. The texts from Ukrainian emotive prose, English poetry and the colloquial style are researched with the help of the data clustering method and the Student’s t-test.
A combination of the machine learning and the classical methods – the data clustering and the Student’s t-test is proposed for text differentiation on the lexical level and the phonological level. The research is done according to the following algorithm.
1. Change uppercase to lowercase of all the letters in the researched Ukrainian and English texts of equal size 2. Remove all the punctuation marks 3. Leave only one space between the words 4. Put a space at the beginning and at the end of the text 5. Calculate the absolute frequency of occurrence of function words 6. Use the method of hierarchical clustering [19] 7. Transcribe the English texts 8. Form samples of equal size for consonant phonemes 9. Calculate the absolute and the mean frequency of occurrence for consonants 10. Form eight consonant phoneme groups 11. Perform the Pearson’s normality test for eight consonant phoneme groups:
N (i npi )2 ˆn2 i1 npi where N is a number of intervals [20 – 21].
12. Perform the Student’s t-test: t ( ) / s n m nm t ;(nm2) , (2) where and are the mean frequencies of occurrence of consonant phoneme groups for the compared samples n and m [22 – 24]. 2.2.
A combination of the data clustering and the Student’s t-test is the basis of the program for text differentiation. The structure of the program includes the following modules [25].
Module of data input/output Module of forming samples of Ukrainian function words Module of calculating the absolute frequencies of function words Module of performing the data hierarchical clustering Module of forming samples of English consonant groups Module of calculating the absolute and the mean frequencies for consonants Module of performing the Pearson’s test
Module of performing the Student’s t-test
The structure of the classes of the software is the following: Main, SampleProcessor, TranscriptionProcessor, ConsonantProcessor, ConsonantUtils, StatisticProcessor.
In the class Main, the text files are downloaded and the sequence of operations is controlled. In the class SampleProcessor, all unnecessary symbols are removed.
In the class TranscriptionProcessor, the English texts are transcribed.
In the class ConsonantProcessor, the samples of consonants are formed.
In the class Consonant Utils, the absolute and the mean frequencies for consonants are calculated. In the class StatisticProcessor, the Pearson’s test and the Student’s t-test are performed. The program code is the following: >library(readxl) >x=read_excel("C:/Users/Катя/Desktop/mag/clust.xlsx") >z=c("London (Before Adam)","Henry(The Sea-Wolf)","Henry(The last leaf)","London(White fang)","Henry(The furnished room)","London(Advanture)") >rownames(x) = z >XDYST=dist(x, method="euclidean") >tree=hclust(XDYST, method="single") >plot(tree) >tree=hclust(XDYST, method="complete")
The Python program code for the literary work “Tsyklon” by O. Honchar is presented in Figure 1. Single Linkage and Complete Linkage are used for a distance between the clusters. Euclidean distance is used for a distance between the objects of the clusters. Complete Linkage is used for the texts of Ukrainian emotive prose in the case Single Linkage is not successful.
The algorithm of the program functioning for the text differentiation by the data clustering and the Student’s t-test is shown in Figure 2.
The data clustering has been performed in eight samples from Ukrainian emotive prose. These are the texts from the following literary works: “Tsyklon” by O. Honchar, “Sobor” by O. Honchar, “Lev ta mysha” by L. Hlibov, “Konyk strybunets” by L. Hlibov, “Malyy Myron” by I. Franko, “Na loni pryrody” by I. Franko and “Zakhar Berkut” by I. Franko. In these comparisons, the authorial style effect is determined. The results of the data clustering for the mentioned literary works are shown in Figure 3.
In Figure 3, we see that the results of the data clustering are successful for “Tsyklon” and “Sobor” by O. Honchar, “Lev ta mysha” and “Konyk strybunets” by L. Hlibov, “Malyy Myron” and “Na loni pryrody” by I. Franko, but not very successful for “Zakhar Berkut” by I. Franko. All the researched literary works by I. Franko are not in the same cluster. Therefore, we change the used Single Linkage for Complete Linkage (Figure 4).
The matrix of distances is shown in Table 1. In this Table, we can see that there is a little distance between two literary works by I. Franko – “Malyy Myron” and “Na loni pryrody”. This result proves that the two literary works have the same author.
The analysis of the comparisons of literary works “Lev ta mysha” and “Konyk strybunets” by L. Hlibov shows a little distance – 11,9. A greater distance is for the comparison “Tsyklon” and “Sobor” by O. Honchar – 16,6. The greatest distance – 28,8 is for the comparison of literary works by different authors – “Na loni pryrody” by I. Franko and “Konyk strybunets” by L. Hlibov.
In Figure 4, we see that the use of Complete Linkage has given a better result, as all the literary works by one author (“Malyy Myron”, “Na loni pryrody” and “Zakhar Berkut” by I. Franko) are in one cluster. Consequently, the use of Complete Linkage is more efficient for solving this task.
The whole process of the data clustering is presented in Table 2.
The task of text differentiating has also been done on the phonological level with the help of the classical method – the Student’s t-test. The English texts – Th. Moore’s poetry and the colloquial style have been differentiated in eight consonant groups. The essential differences between the text compared are shown in Tables 3, 4. level; x is the mean value of frequencies of occurrence of consonant groups; xi x 2 is a sum of squares of difference of the value of middle of the interval and the mean value of frequencies of occurrence of consonant groups, x1 x2 is the value of difference between the two compared samples.
In an unidentified position, the applied Student’s t-test has given a very good result: the essential differences have been established in six out of eight consonant groups. For the groups of the labial and sonorous consonants the differences are statistically insignificant. The mentioned degree of similarity can be explained by the use of words from the colloquial style in the researched Moore’s poetry.
The results have been obtained with a test validity of 95% in the comparisons presented in Tables 3, 4, 5 and 6.
In the position at the beginning of a word, the results are also good (Tables 5, 6). Statistically significant differences have been revealed in five out of eight consonant groups. In addition to the labial and sonorous consonants, the differences are statistically insignificant for the nasals.
Having analyzed the results of this research, we can state that both the data clustering method and the Student’s t-test are efficient for text differentiation on the phonological and lexical levels. However, the former is simpler, the latter is more reliable.
The use of the machine learning method – the data clustering and the classical method – the Student’s t-test has solved the task of text differentiation, the practical application of which is authorship attribution. The proposed combination of the data clustering method and the Student’s t-test is the novelty of the research. The text differentiation task has been successfully done on the lexical level. The texts by I. Franko, O. Honchar and L. Hlibov have been analyzed. The established little distance between the researched texts has proved the fact that they are written by the same author. Consequently, the authorial style effect has been revealed. A good example is the comparison of “Malyy Myron” and “Na loni pryrody” by I. Franko in which the distance is equal to 13,4. The applied classical method – the Student’s t-test has given a good result for determining the style factor effect on the phonological level. The texts of Th. Moore’s poetry and the colloquial style differ in 6 out of 8 consonant groups for an unidentified position in a word and in 5 out of 8 – for the position at the beginning of a word. The results of the research have shown that the data clustering is a simpler method if compared to the Student’s t-test. It shows better results if Complete Linkage is used. However, the Student’s t-test ensures more reliable data with a test validity of 95%. The practical application of the results is the style and authorship attribution. In our future research, another combination of the machine learning methods and the classical methods will be tested for text differentiation.
[13] I. Khomytska, V. Teslyuk, N. Kryvinska, I. Bazylevych, Software-Based Approach towards Automated Authorship Acknowledgement—Chi-Square Test on One Consonant Group. Electronics, 9, 1138, (2020). https://doi.org/10.3390/electronics9071138. [14] A. V. Doroshenko, Application of global optimization methods to increase the accuracy of classification in the data mining tasks, in CEUR Workshop Proceedingsthis link is disabled, 2019, 2353, р. 98–109. [15] A. Doroshenko, R. Tkachenko, Classification of imbalanced classes using the committee of neural networks, in International Scientific and Technical Conference on Computer Sciences and Information Technologies, CSIT 2018, 1, pp. 400–403. (2018). [16] V. S. Pеrebyjnis, Statystychni metody dlia lingvistiv. Nova Knyha: Vinnytsia, Ukraine, (2013). (in
Ukrainian). [17] M. A. Boukhaled, J.-G. Ganascia, Using function words for authorship attribution: Bag-of-words vs. sequential rules, in the 11th International Workshop on Natural Language Processing and Cognitive Science, Oct 2014, Venice, Italy, De Gruyter, Natural Language Processing and Cognitive Science Proceedings, pp. 115–122. (2015). Available online: https://hal.sorbonneuniversite.fr/hal-01198407/document [18] I. Khomytska, V. Teslyuk, Authorship Attribution by Differentiation of Phonostatistical Structures of Styles, in International Scientific and Technical Conference on Computer Sciences and Information Technologies, CSIT 2018, 2, pp. 5–8. (2018). [19] V. O. Klymchuk, Klasternyy analiz.: vykorystannia u psykholohichnyh doslidzhenniah,
Praktychna psykhologia ta sotsialna robota, №4, p. 30–36. (2006). (in Ukrainian). [20] P. C. Gomez, Statistical Methods in Language and Linguistic Research. University of Murcia,
Spain (2013). [21] A. Kornai, Mathematical Linguistics. Springer (2008). [22] V. M. Turchyn, Matematychna statystyka. Navch. Posib. Vydavnychyj tsentr “Akademia”: Kyiv,
Ukraine, (1999). (in Ukrainian). [23] I. Khomytska, V. Teslyuk, I. Bazylevych, I. Shylinska, Approach for minimization of phoneme groups in authorship attribution, International Journal of Computing 19(1), 2020, pp. 55–62. [24] G. I. Ivchenko, Yu. I. Medvedev, Matematicheskaya statistika. Moskva: Vyssh. Shk., p. 248 (1984). [25] A. Batyuk, V. Voityshyn, V. Verhun, Software Architecture Design of the Real-Time Processes Monitoring Platform, in: Proceedings of the IEEE Second International Conference on Data Stream Mining & Processing, DSMP 2018, Lviv, Ukraine, 2018, pp. 98-101. doi: 10.1109/DSMP.2018.8478589