<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Thematic Tagging of Literary Fiction: The Case of Early 20th Century Russian Short Stories</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Young Scientists Symposium</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>St. Petersburg State University</institution>
          ,
          <addr-line>7/9 Universitetskaya emb., 199034 St.Petersburg</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2020</year>
      </pub-date>
      <fpage>0000</fpage>
      <lpage>0002</lpage>
      <abstract>
        <p>The paper deals with the problem of thematic tagging in works of fiction, Russian short stories of the early 20th century (1900-1930) serving as research data. The very concept of discourse theme, or topic, is argued to be fuzzy and ambivalent, all the more so in the case of literary prose. In the present study, theme is conceived as a set of keywords basically (but by no means exhaustively) defining the story's plot. A list of 89 themes was empirically formed, embracing a wide range of topics. A sample corpus of 310 stories was manually tagged, with each story being mapped onto a set of themes. This corpus was divided into three parts corresponding to three periods, 1900-1913, 1914-1922, 1923-1930. These periods of Russian history being radically different, the stories' content varies greatly, too, not only in what concerns political and social themes, but also in quite personal and mundane matters. The paper traces the themes' frequency rates across the three periods, accounting for their change dynamics in terms of sociopolitical context. The sample corpus will be further used as training data in devising computational techniques for automated thematic tagging of literary fiction.</p>
      </abstract>
      <kwd-group>
        <kwd>literary corpus</kwd>
        <kwd>thematic tagging</kwd>
        <kwd>discourse theme</kwd>
        <kwd>Russian literature</kwd>
        <kwd>short stories</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>Introduction</title>
      <p>Thematic tagging is an essential kind of annotation used in digital libraries and large
national corpora. Thus, in Russian National Corpus non-fiction texts are assigned tags
roughly corresponding to manifold spheres of human life, e.g. politics, economics,
law, religion, medicine, business, art, etc. Fiction texts, however, are provided solely
with tags indicating their genre and chronotope.</p>
      <p>This is by no means accidental. The concept of topic, or theme, in literary prose is
an under-studied and problematic issue. The lack of a practicable definition has a
negative effect on developing literary text corpora. In large-scale corpora, texts have
to be thematically annotated but the very principles guiding such annotation are not
clear. There is no agreement on what constitutes a theme in works of fiction, or
whether a text should have a single theme or may be assigned a few of them. Nor is
there a fixed list of themes to choose from. Progress in digital humanities (and
artificial intelligence, in general), though, crucially depends on semantic text processing.</p>
      <p>Copyright ©2020 for this paper by its authors.</p>
      <p>Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).</p>
      <p>In what follows, an individual attempt at manual thematic tagging of a literary texts
sample is presented. It will be further tested against similar annotations performed by
other researchers on the same data, to check out similarities and differences and
together work out a final annotation of the sample.</p>
      <p>The resulting tagged sample will be used twofold. First, it will serve as training
data for a learning model to be used in annotating a larger corpus of Russian short
stories dated between 1900-1930. Second, it will be set against the automatic topic
modeling results, to assess the latter’s adequacy and thus the prospects of using standard
topic modeling algorithms for annotating the above-mentioned corpus. Once a reliable
thematic tagging procedure is found, it can be extended to other literary corpora and
text collections.
1</p>
    </sec>
    <sec id="sec-2">
      <title>Russian Short Stories Corpus (1900-1930)</title>
      <p>The paper deals with recent research within the ongoing project “The Russian
language on the edge of radical historical changes: the study of language and style in
prerevolutionary, revolutionary and post-revolutionary artistic prose by the methods of
mathematical and computer linguistics (a corpus-based research on Russian short
stories)” [1;2]. The project aims to give a comprehensive account of early 20th century
Russian short stories from the viewpoint of genre theory which presupposes their
thematic, structural, and linguistic analysis [3: 237]. Although the project’s particular
emphasis is on language and style, it seems impossible to research into them without
taking account of the short stories’ themes and plot structure.</p>
      <p>
        The first three decades of the 20th century in Russia were a tumultuous time
marked by a series of dramatic historical events (Russo-Japanese war, World War I,
February and October revolutions, the Civil War) that radically changed the course of
Russian history and made an impact on Russian literature and language style. Thus, in
a closer perspective, the project’s goal is to trace the way all the three aspects
(themes, narrative structure and language style) kept changing over time by
comparing different chronological periods [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
      </p>
      <p>To accomplish this, a text corpus was created, containing several thousands of
short stories written in Russia and later, the Soviet Union, from 1900 to 1930 and
published in literary journals or story books. This timespan is divided into 3 parts,
1900–1913, 1914–1922 and 1923–1930, the first covering the time before the great
cataclysms, the second embracing World War I, February and October revolutions
and the Civil War, and the third corresponding to the post-war socialist period. Thus
there are peace periods at the beginning and at the end linked by an intermediate
wartime period.</p>
      <p>
        Each author is represented by a single, randomly selected, story per period. To
ensure robustness of the results, the project aims to account for as many professional
writers as possible, both famous (e.g. Anton Chekhov, Leo Tolstoy, Ivan Bunin,
Maxim Gorky) and lesser-known ones, metropolitan and provincial alike [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ].
      </p>
      <p>
        From the text corpus, a random sample was taken, containing 310 stories by 300
authors (some writers featuring in more than one period, this accounts for a slight
discrepancy in numbers) [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. This sample serves as an initial testbed for linguists and
literary scholars enabling them to put forward and prove (or disprove) preliminary
conceptions concerning Russian short stories of the early 20th century as a special
genre, with its specific themes, plot structure and stylistic features.
2
2.1
      </p>
    </sec>
    <sec id="sec-3">
      <title>Themes and Their Dynamics of Change</title>
      <sec id="sec-3-1">
        <title>Mapping Short Stories onto Sets of Themes</title>
        <p>Identifying themes in works of literature is a challenging job. Theme, or topic, is a
fuzzy concept allowing for an array of interpretations. This holds for any discourse
type, but it is in fiction that the theme notion is most ambivalent. The main problem is
that literary texts are often heavily laden with implicit meanings, or subtext. A short
story theme may never manifest itself in a particular word or phrase and remain an
unspoken message, thus being quite different from the so-called “word themes” [7:
30-31].</p>
        <p>
          In this respect, literary prose is opposed to both academic and mass media
discourse [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. It is no accident that the merits of various information extraction and data
mining techniques are demonstrated on these discourse types rather than on works of
fiction. Thematic content of literary texts by and large defies automatic detection
although certain noteworthy advances have been made in recent years [9;10].
        </p>
        <p>Another difficulty concerning the thematic content of a literary text is that it
normally contains a handful of themes, like love, war, death and desolation, or, say, art,
poverty, and suicide. In fiction, unlike some other text types, themes are not
hierarchically arranged, so that one cannot confidently tag one of them as dominant, or global,
and others as subsidiary, or local. Rather, themes may come in a bunch or be
consequentially arranged, being linked by causal relations.</p>
        <p>In theory, a story’s themes can all be put together in a single proposition, as
suggested for discourse topics in general [11: 134ff], e.g. A poverty-stricken artist
desperately needs money and, unable to sell his paintings, commits suicide. Obviously
enough, each story then will have an individual topic and there will be little chance
for finding regularities.</p>
        <p>
          In the present paper, thematic tagging presupposes the identification of all semantic
components that contribute to the plot, determine the protagonist’s motives and
actions and directly bear on the conflict and its resolution. Themes are considered akin
to keywords [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ], so that each story is mapped onto a clustering of themes. A parallel
can be drawn here with componential analysis that aims to present word meaning in a
bundle of semantic features. The difference, though, is that while componential
analysis tries its best to bring out the total semantic content of a word, a set of themes is not
meant to fully define the short story plot.
        </p>
        <p>As there are no universally accepted algorithms of themes detection in works of
fiction, a careful qualitative analysis is needed at the outset. Topic modelling
techniques [13; 14] will be considered later, once there is a certain amount of data at hand.
Thematic tagging discussed in the present paper was done manually on the
abovementioned sample comprising 310 stories which will be later used as training data for
a learning model.</p>
        <p>We proceeded as follows. A rough set of themes was drawn from the 1st period
stories. It was subsequently tested against the short stories of the two other periods, with
inevitable corrections, deletions and additions. The final set for the whole sample
currently numbers 89 themes, ranging from political to personal, and from
philosophical to mundane.</p>
        <p>In the next sections, groups of themes are identified and figures are presented
showing their frequency rates over the three periods. One should be aware that the
figures in the tables below show the absolute frequencies of themes, i.e. the actual
number of stories in which a particular theme occurs. However, as the total number of
stories per period is close to 100, these figures can be roughly viewed as percentage.
2.2</p>
      </sec>
      <sec id="sec-3-2">
        <title>Political Themes: Wars and Revolutions</title>
        <p>The initial three decades of the 20th century proved a difficult time in Russian history.
Defeat in the Russo-Japanese war (1904–1905), the subsequent political and social
unrest, World War I, February and October revolutions of 1917, resulting in a radical
transformation of economic, political and social life, and finally the Civil War (1917–
1922) with its long aftermath period could not fail to affect Russian literature. It is but
natural that these events are used in many stories as settings. We treat such political
events as themes if they play a key role in the plot. This is often the case with the war
themes. With the revolutions, however, things are different. In a sense, almost all
stories of the third period and some of the second one could be marked by such tag
since their plot would be deemed unrealistic had not the revolutions taken place.
Nevertheless, we think it completely unnecessary to introduce a special tag for the
February bourgeois democratic revolution as it was but a prelude to a much more radical
socialist revolution in October. As for the latter, only a couple of stories, specifically
highlighting the role of this event for the plot, are tagged accordingly.</p>
        <p>It is important to note that themes pertaining to the political context are likely to be
evoked in fiction long after the events concerned. Thus, the Civil War is a theme in
twice as many stories of the third period as those of the second one. The greater the
event, the stronger the postponed effect. One should remember this when comparing
the figures.</p>
        <p>Together with wars and revolutions are listed themes directly bearing on them,
including pre-revolutionary unrest (strikes, military mutinies, peasant riots), political
activities which ultimately led to the revolution (revolutionary movement), death in
the war and execution (Table 1).</p>
        <p>The disastrous Russo-Japanese war failed to get a wide coverage in fiction,
probably because of its local character, or the shame of the defeat. It was largely forgotten
by the time a new war broke out. World War I dragged inconclusively for three and a
half years, causing heavy losses in the Russian Army. Soon after the October
revolution Russia signed Brest-Litovsk Treaty and emerged from the war. World War I is a
frequent theme in the 2nd period stories, but seems no longer significant in the 3rd
period ones.</p>
        <p>The October revolution had brought about so fundamental transformations in all
spheres of life that World War I began to look like a remote and much less topical
event. This accounts for a sharp drop in frequency rate, from 21 to 1. The number of
stories about pre-revolutionary activities also declined.
The impact of the Civil War, in contrast, is on the rise, from 12 during the war to 24
after the end. This is by no means surprising. The war’s outcome was crucial for the
socialist regime, determining the future life of the whole country. In the decades to
come, the Red Army victory would be praised and glorified in numerous works of
fiction, movies, etc.</p>
        <p>The death-in-the-war theme reaches its peak in the 2nd period stories, which is but
natural, given that wars went on over the whole period, and then decreases. The
execution theme, however, gradually goes up, probably due to an increase in the number
of stories dealing with the Civil War.
2.3</p>
      </sec>
      <sec id="sec-3-3">
        <title>Economic and Social Developments</title>
        <p>A thematic block associated with the country’s industrial and social policy became
particularly prominent after the end of the Civil War, during the third period (1923–
1930). Below, in Table 2, major issues are covered.</p>
        <p>The construction of Trans-Siberian railway between 1891 and 1916 was a strong
impetus toward migration over the Ural mountains into Siberia. At the turn of the 20th
century, resettlement was promoted and sponsored by the Russian government.
However, the process was by no means easy and smooth. Disappointment, frustration and
desperation plagued many settlers who found it difficult to survive in the new lands
and were on the verge of giving up the whole idea and going back. These attitudes
appear in three stories of the first period. During World War I, the resettlement
program was dropped. After the end of the Civil war, it was subsumed by larger-scale
projects aimed at the total reconstruction of the country’s life.</p>
        <p>In economics, such large-scale projects were meant to promote technical
modernization and inventions. Hence, a noticeable rise in the number of stories tackling these
issues during the third period.
In the social sphere, the same period is marked by the policy of mass education and
women’s emancipation, the corresponding figures being higher than previously. The
concept of bright future is gaining ground due both to the Soviet propaganda efforts
and the universal human need to believe in a better tomorrow which was particularly
acute after the end the Civil war. The 3rd period stories tend to focus on the young,
active and romantic people, architects of the future happy society.</p>
        <p>The October revolution totally destroyed the former structure of the Russian
society and its old ways of life. The process of instituting a new social order is a key theme
of the third period, hitting a record 38 stories. Sometimes the new order is explicitly
set against the old one, with the former always evaluated positively and the latter,
negatively. Such a neat divide is due to the fact that people disapproving of the
October revolution and the subsequent transformations either had to leave the country or
keep silent. It was impossible for writers denying the new ideas and values to get their
work published.</p>
        <p>Changes inflicted by the revolution affected every person and every family (Table
3). In particular, they totally eliminated the familiar pre-revolutionary pattern of the
well-to-do families dwelling in the large cities in winter and moving to their
countryside estates in summer. The opposition of city and country life is no longer relevant.
Instead, the characters of the 3rd period stories either reside in the city (workers,
clerks) or, most often, are to be found in the rural settlements. There they are trying to
survive in the absence of food, cattle, seeds, agricultural implements, horses or any
other facilities. The peasant life was undergoing major transformations, perhaps the
greatest of all, hence the skyrocketing frequency of this theme in the third period.</p>
        <p>The concept of land is central to the third period, too, as one of the first Bolshevik
decrees, Decree on Land, abolished private property and proclaimed that peasants
should divide up rural land among themselves. This decree led to a split between the
poorer farmers who supported the revolution and sided with the Red Army in the
Civil War and the wealthier ones, who did their best to retain the traditional lifestyle.
The two groups fought over land and the new ways of things in general, sometimes
with violence. The latter theme is increasingly present in short stories across the three
decades.
The number of stories about the non-peasant work and the working class (another
target audience of the early Bolshevik decrees) also grows, the dynamics, though,
being less spectacular than with the peasantry.</p>
        <p>Surprisingly, quite a number of stories in the second and particularly the third
period are about country life, pets and animals (embracing the love of nature, the unity of
man and nature, and hunting). This bunch of themes seems to have been a last resort
for writers hating to praise the Soviet State’s new policies and eager to step aside.
2.4</p>
      </sec>
      <sec id="sec-3-4">
        <title>Religion</title>
        <p>As the Bolsheviks explicitly pursued to eliminate religion with the goal of
establishing state atheism, noticeable change can be seen in the relative frequency of such
themes as Christian God (incorporating the concepts of faith, saints, sin and even
devil) and religion as a social institution across the second and third periods (Table 4).
In the wartime period from 1914 to 1922, the concept of God, quite naturally, was
quite prominent. After the ultimate victory of the Red Army, a peace time ensued,
marked by an active anti-religious campaign launched by the Soviet government.
Spiritual issues are seldom mentioned in the literature of the third period. This is not
the case, however, with religion as a social institution. Although from the quantitative
viewpoint the third period looks exactly as the second one, the situation is different in
two respects. First, in the 3rd period stories, the Christian church no longer enjoyed the
monopoly and had to make room for the Jewish and Buddhist religions. Second, the
references to the church, priests, worshippers, etc. became outright derogatory or
ironic, at best. This fact, incidentally, warrants an interpretative, rather than a purely
quantitative, approach to the semantic analysis of literary fiction.</p>
      </sec>
      <sec id="sec-3-5">
        <title>Personal matters</title>
        <p>One might think that there are timeless, core values in the human life, unlikely to be
affected by political whirlpools and social-life transformations. This may well be so
as regards individual lives, but in the literature, the impact of dramatic public events
prevails. As a result, strictly personal themes like love, marriage, unfaithfulness,
jealousy, children, parental love in the second period shifted out of focus and gradually
declined, becoming less prominent and frequent (Table 5).</p>
        <p>Interestingly, the sexual aspect of love and, more broadly, the body life (including
accidental affair, sexual desire and lust) is on the rise. The same holds for rape.
Prostitution theme vanished from the 3rd period stories following the attempts of the Soviet
government to combat and repress this social practice.
Other human relationships embrace fraternity and solidarity, friendship, mentorship,
and rivalry. Together with them are listed ill effects of interpersonal dealings,
including envy, greed, deceit, treachery, and revenge (Table 6).</p>
        <p>The dynamics of the fraternity and friendship themes reaching their maximum in
the 2nd period stories is quite understandable. In a war, mutual help and reliability are
greatly appreciated and needed and thus it is but natural that stories written during the
war should profile these aspects of human life.</p>
        <p>Revenge-theme frequencies display an opposite pattern with an absolute minimum
in the second period. Rivalry slightly increases over time, while envy, greed and
deceit stay roughly at the same level. Treachery (not to be confused with treason!)
steeply goes up in the 2nd period stories and remains at the same point in the 3rd period
ones. Figures for the mentorship theme rise, reaching roughly 5% in the 3rd period
stories. Their rather high frequency rate may be due to the fact that in Soviet times
this type of relationship was actively promoted in professional and ideological
domains.</p>
        <p>Poverty and hunger plagued people’s life more or less steadily. In the wartime
period, hardships obviously increased. They did not end after the Civil War as the
country was exhausted and near ruin. The economy was devastated, people were starving
and dying from epidemics and lack of health-care. The number of stories highlighting
the contrast between the rich and the poor and the crucial role of money went down in
the 3rd period stories, as there no longer remained wealthy people (Table 7).
The alcoholism theme gradually declines over decades which must be due to
prohibition introduced in Russia at the beginning of World War I. It carried on through the
turmoil of the revolutions and the Civil War until 1925.</p>
        <p>The difference between the times of war and peace is most obviously reflected in
the figures related to the death-in-the-war theme (see Table 1 above). Surprisingly,
there is yet another thematic marker of peace times, and that is boredom. In the epoch
of cataclysms, people do not have the luxury of monotonous everyday life.
Some stories touch on the characters’ inner life (Table 8). The frequency rates of such
themes across the periods can be accounted for by two factors. One is the Soviet
ideology that fundamentally “reformatted” the former social structure, overturning the
system of norms and values and abolishing charity institutions and philanthropy on
the false pretext of an all-sufficient state.</p>
        <p>Another is the above-mentioned shift of focus, both in real life and fiction. In the
aftermath of the Civil War, people haunted by poverty, starvation, homeliness, and
misery could not afford to brood over being lonely, but rather had to survive. The
same applies to hopes and illusions.
The mysticism theme enjoys a rise during the war and drops sharply in the Soviet
period, stories of spiritual experience being unwelcome by the new regime. The
insanity theme also diminishes. Shame, remorse and passion for life get their small but
steady numbers.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Conclusion</title>
      <p>In this paper, early 20th century Russian short stories are analyzed from a thematic
viewpoint. Drawing on a large corpus currently containing a few thousand stories, a
sample of 310 stories was randomly taken to serve as a testbed for a pilot study. It
was further divided into three periods, 1900-1913, 1914-1922, and 1923-1930.</p>
      <p>To detect themes typical of the whole timespan (1900-1930), an empirical
approach was used. Themes were identified manually in a bottom-up fashion (starting
from individual stories) and then tested against the whole sample. In the end, a set of
89 themes was formed, most of them featuring in the tables above. Tagging was also
done manually, to ensure the reliability of results. Each story thus is mapped onto a
set of themes deemed essential for its content.</p>
      <p>For each theme, its occurrences were calculated in the stories of particular periods
and then the frequency rates were compared. The figures vary greatly across periods
almost for every theme, which is not surprising given the turbulent time the country
was going through. Fundamental transformations affected all spheres of life, from
public to private.</p>
      <p>Short stories published in the third period are quite different from those of the first
period. New themes emerged while some old ones dropped in priority, vanished, or
radically changed in evaluation. However, the 2nd period stories cannot be viewed as a
“bridge” between the literature of the two peace periods. They have a distinct
character of their own shaped by the large-scale political and military events.</p>
      <p>Future research into the Russian short stories thematic tagging will run along
several lines. First, a few annotators will perform independent tagging of the sample, to
test the degree of subjectivity, pinpoint controversial issues and work out guidelines
for tackling them. Second, each theme will be provisionally linked to a set of
semantically associated lexical items (words or collocations). It will be then checked to what
extent each theme manifests itself explicitly via such words (it may be presumed that
themes will vary greatly in this respect). The results will determine the prospects for
their automatic detection in literary texts.</p>
      <p>Acknowledgement. The paper is supported by the Russian Foundation for Basic Research,
project # 17-29-09173 “The Russian language on the edge of radical historical changes: the
study of language and style in prerevolutionary, revolutionary and post-revolutionary artistic
prose by the methods of mathematical and computer linguistics (a corpus-based research on
Russian short stories)”.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Martynenko</surname>
            ,
            <given-names>G.Ya.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sherstinova</surname>
            ,
            <given-names>T.Yu.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Melnik</surname>
            ,
            <given-names>A.G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Popova</surname>
            ,
            <given-names>T.I.</given-names>
          </string-name>
          <article-title>Methodological problems of creating a Computer Anthology of the Russian story as a language resource for the study of the language and style of Russian artistic prose in the era revolutionary changes (first third of the 20th century)</article-title>
          .
          <source>In: Computational linguistics and computational ontologies. Issue 2. Proceedings of the XXI International United Conference ʻThe Internet and Modern Societyʼ</source>
          , IMS-2018, St. Petersburg, May 30 - June 2, 2018 Collection of scientific articles, pp.
          <fpage>99</fpage>
          -
          <lpage>104</lpage>
          (
          <year>2018</year>
          ). [In Rissian].
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Martynenko</surname>
            ,
            <given-names>G.Ya.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sherstinova</surname>
            ,
            <given-names>T.Yu.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Popova</surname>
            ,
            <given-names>T.I.</given-names>
          </string-name>
          , Melnik, А.G.,
          <string-name>
            <surname>Zamirajlova</surname>
            ,
            <given-names>E.V.</given-names>
          </string-name>
          <article-title>On the principles of creation of the Russian short stories corpus of the first third of the 20th century</article-title>
          .
          <source>In: Proceedings of the XV International Conference on Computer and Cognitive Linguistics ʻTEL 2018ʼ</source>
          , pp.
          <fpage>180</fpage>
          -
          <lpage>197</lpage>
          (
          <year>2018</year>
          ). [In Rissian].
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Bakhtin</surname>
            ,
            <given-names>M.M.</given-names>
          </string-name>
          : Verbal Creativity Aesthetics. Moscow, Iskusstvo (
          <year>1979</year>
          ). [In Rissian].
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Martynenko</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sherstinova</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          <article-title>Emotional Waves of a Plot in Literary Texts: New Approaches for Investigation of the Dynamics in Digital Culture</article-title>
          . In: Alexandrov,
          <string-name>
            <given-names>D.</given-names>
            ,
            <surname>Boukhanovsky</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            ,
            <surname>Chugunov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            ,
            <surname>Kabanov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            ,
            <surname>Koltsova</surname>
          </string-name>
          ,
          <string-name>
            <surname>O</surname>
          </string-name>
          . (eds.).
          <source>Digital Transformation and Global Society. DTGS 2018. Communications in Computer and Information Science</source>
          , vol
          <volume>859</volume>
          , pp.
          <fpage>299</fpage>
          -
          <lpage>309</lpage>
          (
          <year>2018</year>
          ). https://doi.org/10.1007/978-3-
          <fpage>030</fpage>
          -02846-6_
          <fpage>24</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Martynenko</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sherstinova</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          <article-title>Linguistic and Stylistic Parameters for the Study of Literary Language in the Corpus of Russian Short Stories of the First Third of the 20th Century</article-title>
          . In: R.
          <article-title>Piotrowski's Readings in Language Engineering</article-title>
          and Applied Linguistics,
          <source>Proc. of the III International Conference on Language Engineering and Applied Linguistics (PRLEAL-2019)</source>
          , Saint Petersburg, Russia, November
          <volume>27</volume>
          ,
          <year>2019</year>
          ,
          <string-name>
            <given-names>CEUR</given-names>
            <surname>Workshop</surname>
          </string-name>
          <article-title>Proceedings</article-title>
          . Vol.
          <volume>2552</volume>
          , pp.
          <fpage>105</fpage>
          -
          <lpage>120</lpage>
          (
          <year>2020</year>
          ). http://ceur-ws.
          <source>org/</source>
          Vol-
          <volume>2552</volume>
          /.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Martynenko</surname>
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Sherstinova</surname>
            <given-names>T.</given-names>
          </string-name>
          <article-title>Corpus of Russian Short Stories of the First Third of the 20th Century: Theoretical Issues and Linguistic Parameters</article-title>
          .
          <source>In: Structural and Applied Linguistics</source>
          . Vol.
          <volume>14</volume>
          .
          <string-name>
            <surname>St</surname>
          </string-name>
          . Petersburg.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Zhirmunskii</surname>
            <given-names>V.M.:</given-names>
          </string-name>
          <article-title>Literary theory</article-title>
          . Poetics, Stylistics. Leningrad,
          <string-name>
            <surname>Nauka</surname>
          </string-name>
          (
          <year>1977</year>
          ). [In Rissian].
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Blummer</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kenton</surname>
            ,
            <given-names>J.M. Academic</given-names>
          </string-name>
          <article-title>Libraries' Outreach Efforts: Identifying Themes in the Literature</article-title>
          .
          <source>Public Services Quarterly</source>
          ,
          <volume>15</volume>
          (
          <issue>3</issue>
          ),
          <fpage>179</fpage>
          -
          <lpage>204</lpage>
          (
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Rhody</surname>
            ,
            <given-names>L.M.</given-names>
          </string-name>
          <article-title>Topic Modeling and Figurative Language</article-title>
          .
          <source>Journal of Digital Humanities</source>
          ,
          <volume>2</volume>
          (
          <issue>1</issue>
          ) (
          <year>2012</year>
          ). URL: http://journalofdigitalhumanities.org/2
          <article-title>-1/topic-modeling-and-figurativelanguage-by-lisa-m-rhody/.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Iyyer</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guha</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chaturvedi</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Boyd-Graber</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Daumé</surname>
            <given-names>III</given-names>
          </string-name>
          , H. Feuding Families and Former Friends:
          <article-title>Unsupervised Learning for Dynamic Fictional Relationships</article-title>
          .
          <source>In: Proceedings of the 2016 Conference of the North American Chapter of the Association of the Computational Linguistics</source>
          . (
          <year>2016</year>
          ). URL: https://www.aclweb.org/anthology/N16-1180.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11. van Dijk,
          <string-name>
            <surname>T.</surname>
          </string-name>
          :
          <article-title>Text and Context: Explorations in the Semantics and Pragmatics of Discourse</article-title>
          . Longman, London and New York (
          <year>1977</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          12.
          <string-name>
            <surname>Watson</surname>
            <given-names>Todd</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>R. Discourse</given-names>
            <surname>Topics</surname>
          </string-name>
          . John Benjamins, Amsterdam &amp; Philadelphia (
          <year>2016</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          13.
          <string-name>
            <surname>Blei</surname>
            ,
            <given-names>D.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ng</surname>
            ,
            <given-names>A.Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jordan</surname>
            ,
            <given-names>M.I.</given-names>
          </string-name>
          <article-title>Latent Dirichlet Allocation</article-title>
          .
          <source>Journal of Machine Learning Research</source>
          ,
          <volume>3</volume>
          (
          <issue>4</issue>
          -
          <fpage>5</fpage>
          ). (
          <year>2003</year>
          ). URL: http://jmlr.csail.mit.edu/papers/volume3/blei03a/ blei03a.pdf
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          14.
          <string-name>
            <surname>Mitrofanova</surname>
            ,
            <given-names>O.A.</given-names>
          </string-name>
          <string-name>
            <surname>Probabilistic</surname>
          </string-name>
          <article-title>Topic Modelling of the Russian Text Corpora by Means of GenSim Toolkit</article-title>
          . In: Proceedings of the International Conference «Corpus Linguistics - 2015
          <string-name>
            <given-names>».</given-names>
            <surname>St</surname>
          </string-name>
          .-
          <string-name>
            <surname>Petersburg</surname>
          </string-name>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>