This paper describes automatic topic spotting of literary texts based on the Russian short stories corpus, compiling stories written in the first third of the XXth century. Non-negative matrix factorization (NMF) is a valuable alternative to existing approaches of dynamic topic modeling and it can find niche topics and related vocabularies that are not captured by existent methods. The experiments were conducted on text samples extracted from the corpus, the given samples contain texts of 300 different authors. This approach allows to trace the topic dynamics of Russian prose for 30 years from 1900 to 1930.
In the last decade topic modeling has become one of the most popular issues of computer linguistics. Topic modeling is usually understood as building a model that shows which topics appear in each document [Daud et al., 2010]. The topic model of a collection of text documents determines whether each document belongs to a different topic and it generates a list of words (terms) from which each topic is formed [Blei, Lafferty, 2006]. With this method, it is possible to process large amounts of data (fiction texts, magazine articles, news reports, social media, reviews, etc.) and automatically receive information about the topics of texts. Knowing what people are talking about and understanding their concerns and opinions is very valuable for science, business, political campaigns, etc.
field of research in an unsupervised theory. Probabilistic latent semantic analysis (PLSA) based on the principle of maximum likelihood was one of the first proposed as an alternative to classical clustering methods based on the calculation of distance functions. Next to PLSA latent Dirichlet allocation (LDA) and its numerous generalizations were proposed.
Following on from LDA methods similar probabilistic approaches have been consistently developed to track the evolution of topics over time in a sequentially organized corpus of documents, such as the dynamic topic model (DTM) [Blei, Lafferty, 2006]. Alternative algorithms, such as non-negative matrix factorization (NMF) [Lee and Seung, 1999] considered in this paper, have proven effective in finding underlying topics in text corpora [Wang et al., 2012]. For this reason, that algorithm was chosen for this study, which is aimed to automatic selection of dynamic topics in the Russian short stories corpus of the first third of the XXth century [see Sherstinova, Martynenko in this volume]. 2
that aims to detect useful features [Mu¨ller and Guido, 2017]. It is utilized for dimensionality of non-negative matrices, because it decomposes the data into factors in such a way that there are no negative values in them. Therefore, this method can be applied only to those data where features have non-negative values, as a non-negative sum of non-negative components cannot become negative [the same].
ants are used in the modelling process [Darek and Cross, 2016]. In addition, another benefit is that NMF can identify niche topics that tend to be under-reported in traditional LDA approaches [O’Callaghan et al., 2015]. Niche topics are sub-topics that can be identified within a dynamic topic.
The ability of NMF to consider how significant a word is to a document in a text collection, based on weighted term frequency values, is particularly useful. In particular, the application of log-based TF-IDF weighting factor to the data before the construction of the topic model contributed to diverse but semantically coherent topics that are less likely to be represented by the same high-frequency terms [Darek and Cross, 2016]. This makes NMF a suitable method for identifying both broad groups of high-level documents and niche topics with specialized dictionaries [O’Callaghan et al., 2015]. 3
non-negative matrix factorization to the Russian short stories corpus of the first third of the
first third of the XXth century, which is developed at the Philology Department of Saint Petersburg State University in cooperation with Philology Department of the National Research
nova, Martynenko, 2019]. The data set consists of 300 stories of 300 unique writers both world-famous and barely known. The corpus is a homogeneous resource, which is focused on one of the most common genres of fiction the short story. This genre is the most popular among prose writers, its presence may be found in almost all kind of literary of almost all writers.
The corpus under development covers one of the most dramatic periods in the development of the Russian language and literature. The central point that divides the first third of the twentieth century into different time periods is the October Revolution of 1917. All other events are considered either as leading to it or as arising from it. It allows to make the quantitative analysis of language changes in rather wide chronological frameworks and to estimate what of the arisen language changes were fixed in language, were started to be often used by speakers or were disappeared after the revolutionary epoch [Martynenko et al., 2018b].
The base of the corpus provides a means for exploring the language of the first third of the twentieth century (1900-1930) divided into three main periods: 1) the beginning of the XXth century and the prerevolutionary years, including the First world war, 2) the revolutionary years the February and the October revolutions and the Civil war, and 3) postrevolutionary years from the end of the Civil war to 1930. Each of these time periods will be analyzed separately and the results will be combined into an overall picture, reflecting the development of the Russian language in the first third of the XXth century [Martynenko et al., 2018b]. 5
the document length are applied before each matrix is written. It includes marking documents and creation of a document matrix for the window topic where the topic model is created by applying NMF to each time window.
Determining the number of topics is a nontrivial task because the choice of too few of them leads to overly generalized results while choosing too many topics entails too many small, highly similar topics [Green and Cross, 2015]. For the cases when this number is not known in advance there are different strategies for automatic or semi-automatic selection of number of topics. In particular it is proposed to build a Word2Vec Skipgram model using the Gensim library (https://radimrehurek.com/gensim/) from all documents in the case. The TC-W2V measure is used to compare different topic models and then select a model with a suitable number of topics. More details on the TC-W2V are in [O’Callaghan, 2015].
diverse but nonetheless coherent topics that are less likely to be represented by the same high-frequency terms allowing identification of both broad and niche topics with specialized vocabularies [O’Callaghan et al., 2015]. In the context of the study of the first third of the
components of the topics and analyze the realities of the period in more detail. To illustrate this idea table 5 shows the top 10 terms for 4 dynamic topics. Terms in bold are unique to a topic; terms in italics are met in an overall description of a topic and in time windows (or even in one time window), terms in bold and italics are found within time windows but are not in an overall description of a topic.
6
The highest interest for linguistic analysis is the content of dynamic topics. Table 5 lists the top 4 dynamic topics penetrating all time periods (1900–1913, 1914–1922, 1923–1930). Table 6 shows niche topics and vocabularies of each dynamic topic in a specific time period. For instance, the first broad topic in the first time window is represented by 40 words with the biggest amount of unique terms (писать (to write), любовь (love), любить (to love), сцена (scene), роль (role), ребенок (child), муж (husband), жена (wife), счастье (happiness), кабинет (room, office), деньга (money), русский (Russian), пароход (steamer), город (city, town) and etc.). Inference should be drawn that writers at the beginning of the century wrote a lot about a mode of life: about family (муж (husband), ребенок (child), жена (wife), мама (mom), сестра (sister), отец (father)), work (кабинет (room, office), деньга (money)) and events that occurred with the main characters, which interacted with people of different professions (купец (merchant), приказчик (manager), извозчик (horse-cab driver), доктор (doctor)). The number of unique terms is not numerous in the second time window (1914–1922), which is due to the fact that this is a revolutionary time and the description of life is minimal. In the post-revolutionary period the vocabulary increases again, there are unique words that reflect the ¾new life¿ (товарищ (comrade), завод (factory), гражданин (citizen), рабочий (working), etc.).
(pond), река (river), ночь (night), солнце (sun), куст (bush), лес (forest), волк (wolf ) ), during 1914–1922 - more abstract (ветер (wind), море (sea), небо (sky), солнце (sun), ночь (night), берег (bank)), 1923–1930 (сосна (pine), птица (bird), зверь(beast), лес (forest), болото (swamp)). The third dynamic topic is filled with words related to the military sphere. However, if to look through the niche topics and vocabularies of each period at the beginning of the century only a few words can be attributed to the military theme (солдат (soldier), офицер (officer), пост (post)), the rest of unique terms are more related to the usual way of life (барин (lord), старик (old man), деревня (village), благородие (honour), etc.)), which can indicate to the maintenance of order and regulation of people relations . In the second time window (1914–1922) two unique words ¾немецкий¿ (German) and ¾немец¿ (German) appeared, and there are no abstract words, almost all content refer to the military (солдат (soldier), офицер (officer), стрелять (to shoot), рота (troop), etc.), which fully reflects the revolutionary period. There is a large number of unique words in the third time window where there are the following niche topics: movement by train (вагон (coach), пассажир (passenger), поезд (train), станция (station), ход (motion), курс (course)), family (муж (husband), мама (mom), ребенок (child), мальчик (boy)), house/home (дом (house, home), кухня (kitchen)). Only the word ¾солдат¿ (soldier) can be attributed to the military topic.
народ (folk), изба (hut, house), etc.), religion (поп (priest), батюшка (priest), церковь (сhurch). It is worth paying attention to the dynamics of changes in the religious topic: more words in the revolutionary time (батюшка (priest), Бог (God), святой (saint)) compared to the beginning of the century (батюшка (priest)), and the postrevolutionary period (поп (priest), церковь (сhurch)).
If we consider the components of topics from the linguistic viewpoint the largest number of words belongs to the nominative class, which is represented by common nouns. Proper names (Александр (Alexander), Алексей (Alexey), Анна (Anna), Владимир (Vladimir), Володя (Volodya), Мишка (Mishka), Вера (Vera), etc.) are deliberately removed since there are many dialogues in the data. The frequency of names is very high and its topic distribution is not conditioned by anything.
Most nouns refer to the description of people (ребенок (child), девушка (young lady), женщина (woman), старик (old man), дед (grandfather, old man), баба (country woman, peasant’s wife), отец (father), мать (mother), муж (husband), жена (wife), father (батюшка), матушка (mother), сестра (sister), дядя (uncle), etc.), profession (солдат (soldier), барин (lord), офицер (officer), студент (student), крестьянин (peasant), доктор (doctor), etc.), body parts (рука (hand, arm), нога (leg), грудь (chest)). Another group of nouns - everyday realities (комната (room), улица (street), пост (post), изба (hut, house), дом (house, home), письмо (letter), etc.), nature and animals (лес (forest), куст (bush), волк (wolf ), солнце (sun), лошадь (horse), pond (пруд), река (river), ночь (night), снег (snow)), abstract (жизнь (life), счастье (happiness), мысль (thought), смерть (death), etc.), as well as collective nouns (толпа (crowd), рота (troop), народ (folk)).
The predicative class is represented by verbs: писать (to write), знать (to know), любить (to love), хотеть (to want), стоять (to stand), кричать (to shout), бежать (to run), думать (to think), обедать (to dine), прийти (to come), сидеть (to sit), жить (to live), работать (to work), спать (to sleep), глянуть (to peep), стрелять (to shoot), вскочить (to jump up), пойти (to go). From the point of view of semantic classification of verbs developed by V. V. Vinogradov and supplemented by G. A. Zolotova [Zolotova, 2004] the present verbs belong to the main semantic classes:
1) verbs of movement: стоять (to stand), сидеть (to sit), глянуть (to peep), бежать (to run), прийти (to come), вскочить (to jump up), идти (to go); 2) verbs of speech action: кричать (to shout); 3) verbs of mental actions: знать (to know), думать (to think), казаться (to seem);
5) verbs of physiological action: жить (to live), обедать (to dine), спать (to sleep); 6) verbs of activity or occupation: работать (to work), писать (to write), стрелять (to shoot); 7) modal verb: хотеть (to want).
(big), черный (black), темный (dark), горбатый (humpbacked), старший (senior)) and relative adjectives (рабочий (working), русский (Russian), немецкий (German)).
tagmatic relations that organize the text [Mitrofanova et al., 2014; Mitrofanova, 2014]. The language connections within the topics may be describes with lexical functions in the model ¾Meaning < = > Text¿ [Melchuk, 1974/1999] which allows to cover the predictable, idiomatic connections of the word and its lexical correlates.
Among paradigmatic relations in topics the following are prevailed: synonymy (Syn), antonymy (Anti) and derivational (Der) relations, etc. For example, Syn: мама (mom) мать (mother), друг (friend) товарищ (comrade), фабрика (plant) завод (factory), черный (black) темный (dark), поп (priest) батюшка (priest), etc. Anti: деревня (village) город (city, town), Der: работа (work) рабочий (working) работать (to work), немец (German) немецкий (German), любовь (love) любить (to love), крик (shout) кричать(to shout), команда (command, team) командир (commanding officer), etс. Partitive relations: семья (family) ребенок (child), мама (mom), отец (father), дядя (uncle), сестра (sister), муж (husband), жена (wife), отец (father), мать (mother), тетка (aunt); армия (military) офицер (officer), солдат (soldier), рота (troop), штаб (headquarter), капитан (сaptain), команда (command, team), etc.; природа (nature) пруд (pond), река (river), берег (bank), лес (forest), снег (snow), солнце (sun), ветер (wind), дерево (tree), куст (bush), болото (swamp), etc..; лес (forest) дерево (tree), куст (bush), болото (swamp); охота (hunt) ружье (rifle), зверь (beast), лес (forest), огонь (fire); деревня (village) изба (hut, house), телега (telega, horse wagon), крестьянин (peasant), барин (lord); дом (house, home) комната (room), дверь (door), окно (window), лампа (lamp), кухня (kitchen), кабинет (room, office); передвижение на поезде (go by train) – вагон (coach), пассажир (passenger), поезд (train), станция (station), ход (motion); завод (factory) – работа (work), рабочий (working), работать (to work), машина (machine), etc.
Syntagmatic relations are realized at the level of valence frames filled with words from the topic. Among lexical functions Oper1,2 may be selected, which connect a verb, the name of the first or the second actant in the role of subject and the name of the situation as additions: суп (soup) – обедать (to dine), письмо (letter) - писать (to write), винтовка (gun) – стрелять (to shoot), ребенок (child) – кричать (to shout), etc. In addition, there are a number of examples for the implementation of the lexical function Cap: (команда (command, team) – командир (commanding officer); штаб (headquarter) – начальник (chief ); отряд (squad) – командир (commanding officer), церковь (church) – поп (priest), пароход (steamer) - капитан (captain), etc. The lexical function Equip ("personnel, staff"): people (folk) – man (country man, peasant man), etc., lexical function Doc (res) ("document that is the result"): write (to write) – letter (letter); draw – drawing, etc.
of the theory of lexical functions, as well as to use historical and literary approaches for this purpose. The content of the topics allows to draw conclusions about the topic dynamics of
09173 “The Russian language on the edge of radical historical changes: the study of language and style in prerevolutionary, revolutionary and post-revolutionary artistic prose by the methods of mathematical and computer linguistics (a corpus-based research on Russian short stories)”. [Melchuk, 1974/1999] Melchuk I. A. (1974/1999) Experience of the theory of the linguistic models ¾Meaning Text¿. Moscow, 1974/199. 1974 (In Russ.) = Opyt teorii lingvistitcheskix modelej Smysl Tekst, Moskva, 1974/1999.