=Paper=
{{Paper
|id=Vol-2851/paper33
|storemode=property
|title=Paragraphemic Component of Multimodal Text Analyzed in Software Product Written in Python
|pdfUrl=https://ceur-ws.org/Vol-2851/paper33.pdf
|volume=Vol-2851
|authors=Nataliia Kunanets,Marta Karp,Tetiana Yaremchuk
|dblpUrl=https://dblp.org/rec/conf/itpm/KunanetsKY21
}}
==Paragraphemic Component of Multimodal Text Analyzed in Software Product Written in Python==
https://ceur-ws.org/Vol-2851/paper33.pdf
Paragraphemic component of multimodal text analyzed in
software product written in Python
Marta Karpa, Natalia Kunanetsa,b, Tetiana Yaremchuka
a
Lviv Polytechnic National University 12 Bandera street, Lviv, 79013, Ukraine
b
Ivan Franko National University of Lviv, Universutetska Street 1, Lviv, 79000, Ukraine
Abstract
Scientific and technological progress towards the use of information technology has occupied
all spheres of human activity. Involvement in linguistic research of information technologies
is extremely relevant and popular today, as they provide scientists with a diverse set of
functions for the study of language phenomena at the modern scientific level. Contemporary
literature is evolving with society thus the relevance of the research is caused by the need in
multimodal text analysis using modern technology. Since multimodal fiction prose text has
been full of verbal and nonverbal components, programs of computer technology help to
identify the appropriate component. By dint of the implementation of an IT project from the
elaborating software program there is an opportunity to identify quickly the paragraphemic
component of a multimodal fictional prose text. The main aim of the article is to explore the
research method of multimodal fictional prose text which includes developed software
product written in Python. In the article the explanation how to research different modes
(bold, italics, underlining, strikethrough) of paragraphemic component in multimodal
fictional prose text using developed software product written in Python has been given. The
Jonathan Safran Foer’s novel “Extremely loud and incredibly close” has been used as
application base. Multimodal fictional prose text is a modern phenomenon that combines
verbal and nonverbal components to interpret relevant content through a variety of semiotic
channels of information. Moreover, the perception of the text occurs only with the correct
interpretation of its component functions. It has been defined that the software product
facilitates the identification of the verbal and nonverbal (paragraphemic) components in a
multimodal fictional prose text.
Keywords 1
Multimodal fictional prose text, verbal component, paragraphemic component (bold, italics,
underlining, strikethrough), software product written in Python.
1. Introduction
Scientific and technological progress and the trend towards the use of information technology have
occupied all spheres of human activity. Life became easier. Involvement in linguistic research of
information technologies is extremely relevant and popular today, as they provide scientists with a
diverse set of functions for the study of language phenomena at the modern scientific level.
Contemporary literature is evolving with society, thus a need in text analysis using modern
technology has arisen. Adequate perception and interpretation of printed (electronic) text call for the
addressee has the so-called “visual literacy” – i.e. the ability to read visual, graphic and typographic
configurations and recognize the functions of signs and modes used in the text. Narrative competence
performs the most important function – i.e. the introduction of new information into the stereotypical
consciousness, which helps to break down old stereotypes, the birth of new ones, developing a
creative and chaotic state of consciousness, which like any synergetic system looks for self-
Proceedings of the 2nd International Workshop IT Project Management (ITPM 2021), February 16-18, 2021, Slavsko, Lviv region, Ukraine
EMAIL: martakarp26@gmail.com (A. 1); nek.lviv@gmail.com (A. 2); taniayaremchuk@gmail.com (A. 3)
ORCID: 0000-0002-7332-7739 (A. 1); 0000-0003-3007-2462 (A. 2); 0000-0003-4178-1547 (A. 3)
©️ 2021 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
organization and elimination of destructive tendencies which contributes to the strengthening of
attractive mechanisms and processes, the formation of new semiotic narrative structures. Constant
updating of narrative competence, its permanent balancing leads to the dynamization of the cultural
semiotic universe, reformulation of semantic structures of already interpreted texts and, finally,
ensures the reservation and transmission from generation to generation of culturally significant
information. Yu. Lotman wrote: “in the general system of culture, texts perform two main functions:
the adequate transmission of meanings and the creation of new meanings” [26, p. 81]. One of the
factors in the implementation of these functions is narrative competence. The perception of the text is
facilitated by its structural analysis.
2. Multimodal text vs Information technology
Since multimodal fiction prose text has been full of verbal and nonverbal components, programs of
computer technology help researchers to identify the right component. By dint of the implementation
of an IT project from the elaborating software program there is an opportunity to identify quickly the
paragraphemic component of a multimodal fictional prose text.
2.1. Multimodal novel Extremely loud and incredibly close by J. S. Foer
Let’s consider the functionality of the software product on the example of the text of Jonathan
Safran Foyer’s novel “Extremely loud and incredibly close”. The application of bold, italics,
underlining and strikethrough to the text plays an essential role in the process of the analysis of the
novel. By dint of them, the author emphasizes the important information and creates images in the
novel. It has encоuraged us to develop new software product written in Python language. The
functionality of the software product рrovides the definition of the text selected by the paragraphemic
сomроnеnts and helps to identify quickly the necessary information.
The multimodal text of Jonathan Safran Foyer’s novel "Extremely Loud and Incredibly Close” is
significant in volume and serves as a representative empirical basis that guarantees the reliability and
objectivity of the study of the text, illustrates and confirms the main provisions of the study. It is
difficult to analyze a large amount of material manually and it takes a lot of time that is why a
software product has been developed. It hastens the process of analysis, classification, statistical
calculations, especially on large amounts of information. Linguistic research can significantly
advance due to the quality and structural use of the developed software product.
2.1.1. Analysing paragraphemic component of multimodal text by means of
software product written in Python language
The research process is accelerated and optimized by using the functionality of the developed
software product in Python. The name “Python” was borrowed by developer Guido van Rossum in
1989 from the Monty Python British show and is a powerful, full-fledged, object-oriented
programming language with dynamic semantics. This programming language is attractive for rapid
software development due to its high-level structures, dynamic type of processing and elegant syntax.
It is an ideal tool for creating of applications.
Python can be applied to a wider range of software products, unlike Awk and Perl, but many
Python commands are so easy to use. The advantage of Python is that it allows to break the program
code into modules that can then be used in other software products. Python is associated with a large
library of standard modules that can be used as a basis for new programs. Python assists with saving a
significant amount of time that is usually spent on compilation. Moreover, it helps to experiment with
the capabilities of the language, write code templates or test features in the development of “down-
up”. This programming language allows to create compact and convenient program codes, which are
usually much shorter than similar program codes written in C or C ++, due to: high-level data types
that expresses complex operations with a single instruction [22, p. 250]; grouping instructions are
performed using spacing instead of curly braces; there is no need to declare variables.
The developed software product facilitates and accelerates the process of collecting linguistic
material, which is separated by modes of paragraphemic component (italics, bold, underlining,
strikethrough) in the studied texts. The software product was developed using Python version 3.5.2.
The principle of operation of the software product is to select from the studied texts those fragments
that are separated by modes of the paragraphemic component and displays them in a separate field to
simplify work with them.
The first step after processing text with Abbyy FineReader and before developing the software is
to install the “pythondocx” module, which allows to create and edit documents with the extension
.docx in the MS Word. To install this module, run the command “pip install python-docx” in the
computer console (Fig. 1).
Figure 1: Running the pip install python-docx command
It is of paramount importance to enter “python docx”, not “docx” when the module is installed. At
the same time, import “docx” should be used directly when working with the program, and not
“import python-docx” during installation the “python-docx” module. Files with the file extension
.docx have a developed internal structure. In the python-docx module, this structure is represented by
three different data types. At the top level, the object of Document processing is the entire document.
The Document object involves a list of Paragraph objects, which are paragraphs of the document.
Each paragraph contains a list consisting of one or more Run objects, which composes fragments of
text with different formatting styles: import docx doc = docx.Document('example.docx') # the number
of paragraphs in one document print(len(doc.paragraphs)) # the text of the first paragraph in the
document print(doc.paragraphs[0].text). Besides, MS Word uses two types of styles for documents:
paragraph styles that can be applied to Paragraph objects, and symbol styles that can be applied to
Run objects. Styles can be assigned to both Paragraph and Run objects, taking them to the style
attributes a string value. This line must be a style name. If value None is set for the style, the
Paragraph or Run object will not have a style associated with it.
Paragraph styles: Normal Heading 4 List 2 List Number TOCHeading Body Text Heading 5 List 3
List Number 2 Title Body Text 2 Heading 6 List Bullet List Number 3 Body Text 3 Heading 7 List
Bullet 2 List Paragraph Caption Heading 8 List Bullet 3 Macro Text Heading 1 Heading 9 List
Continue No Spacing Heading 2 Intense Quote List Continue 2 Quote Heading 3 List List Continue 3
Subtitle
Symbol styles: • Emphasis • Strong • Book Title • Default Paragraph Font • Intense Emphasis •
Subtle Emphasis • Intense Reference • Subtle Reference For example: paragraph.style = 'Quote'
run.style = 'Book Title'
Some fragments of text represented by Run objects can be further formatted by using attributes.
One of three values can be specified for each of these attributes: True (attribute activated), False
(attribute disabled) and None (the style set for this Run object is installed): bold – bold text, underline
– underlined text, italics – italic text, strike – strikethrough text.
The import command loads all the necessary modules for working with text. Firstly, a glob library
is imported, which allows to create lists of files using extension templates applied to directories. The
sys module is a module built into the default interpreter program (shell) that provides access to
operations. The sys.path variable is a list of strings that define the way of modules which are used by
the interpreter program. Enter the initial position of FBPos for search, assign it a value of 0. The
Run.italic attribute detects whether the text is formatted in italics, but it does not know if the text
block has a style that is displayed in italics. However, if it is necessary, it can be found by checking
Run.style.name (if it is known that the styles in the document that are under study are displayed in
italics). Then, we access the file, which will be saved in .docx format with the help of ResultFile.
Thus, this file is already open and ready to write new data.
After finishing with the basic steps, we run an outer FOR loop to read the files under study (if
more than one file) and an internal similar FOR to scan the file itself and search for fragments in
italics. When program finds the appropriate values, it writes them to a text file, separating them with
tabs \ t, moves to the next new line for further work and writing. At the end of the inner and outer
loops, the program closes the record file and ends its action (Fig. 2).
Figure 2: Running the inner and outer FOR loops
import glob import sys import docx ModulePath = 'D:\\InstrumentumLingualis\\Python' if
not(ModulePath in sys.path): sys.path.append(ModulePath) import IlGlob 79 InFolder =
IlGlob.SetInput() OutFolder = IlGlob.SetOutput() NamePrefix = IlGlob.SetNamePrefix() FBPos=0
ResultFile = open(OutFolder + NamePrefix + ‘Текст курсивом.docx’), WorkFileList =
glob.glob(InFolder + ‘*.docx’) for FileName in WorkFileList: ShortFileName =
IlGlob.GetShortName(FileName) print(ShortFileName) FBPos = 0 DocumentFile = open(FileName)
DocumentString = DocumentFile.read() for p in ShortFileName.paragraphs: for run in p.runs: if
run.italic temp=run.italic ResultFile.write(ShortFileName + '\t' + temp + '\n') FBPos = EPos +1
ResultFile.close()
Thus, the software product contributes to the study of texts multimodality. In the study of
multimodality, the channel of communication plays a vital role, because it determines the strength of
materiality to comprehend the information provided. There are two definitions of this term in
linguistics: the first one refers to the physical materials used in the transmission of communication
(medium), and the second one can also be defined as a communication channel. However, modes and
communication channels cannot be unambiguously compared. According to linguists G. Kress and T.
van Leuven, a certain semiotic regime can appear in various media [28, p. 40]. For example, language
can be spoken or written. Different modes can be implemented in the same environment, as it is
demonstrated by the use of images and words in comics or stories. Because materiality can function
as a source of difference and also as a meaning, multimodal narrative analysis views the media as one
element that proceeds in a wider range of semiotic regimes and is used in the text. It is less interested
in addressing specific media issues separately. Thus, multimodal narrative analysis can focus on a
single medium (printed literature) or can distinguish between narratives from different media
(audiovisual and written). However, regardless of the number of media types, the focus remains on
the integration of semiotic resources, not only on the comparison of the media [23 p. 203]. The
interrelation between materiality and multimodality draws attention as to the physical work which is
involved in narrative processing, both through the use of modes and technologies, as to the human
body and its senses.
Narrative is not only mode of fictional expression; it is also mode of human talent. Therefore, the
role of sensory modes is of paramount importance in multimodal analysis, because fictional worlds
are created by the use of verbal or visual resources. Addressors and addressees interact with the
essence of the stories in different ways, whether through gestures and the tone of voice in oral stories,
or through sensorimotor manipulations with a page, keyboard, screen or other material [28, p. 45].
The active interaction of semiotic resources affects the content of the story, and also requires a
certain form of transcription as the first step in the process of analysis and interpretation. Multimodal
theory requires to rethink the benefits of any particular mode. Transcription can reveal not only the
verbal content of the story, but also involved range of semiotic resources [20, p.109]. Thus, it is
possible to reveal more clearly the regularities of their integration on various examples of the story
and help to identify points of commonality and contrast in the narrative spectrum. Transcription must
be both not only systematic and repetitive, but flexible enough to cover the rich variety of
multimodality. Although multimodality explains language as only one of many semiotic resources
working in the acquisition of meaning, the use of verbal components still prevails in the practice of
the transcription process [27, p. 140].
However, people not only perceive the information but also interact with text, the physical work
that is done during the interaction with printed pages, digital screens, or computer technology comes
to the fore. Linguists N. Norgaard and A. Gibbons give examples of specific cases where a mysterious
game with printed pages can be used to create additional meaning in modern novels [30, p. 53]. But
the influence of tactile modality is most strongly felt in the context of a text about new media, where
the reader must physically manipulate an element of the digital apparatus (mouse, keyboard, headset)
to understand the story being told. The impact on the reader’s reaction can be various. Scientist M.
Toolan acknowledges the well-documented problems that arise when the digital interface prohibits
readers from immersing themselves in text. The process-oriented nature of multimodal theory is not
limited to the physicality of the text, narrator or audience, but is complemented by cognitive
approaches [18, p. 123].
The software product facilitates the identification of the verbal component in a multimodal
fictional prose text. Up until recently, the verbal component has been considered as the main source of
information in the fictional text. The verbal components of a multimodal fictional prose text include
language code tools: words, phrases, sentences and texts used to convey information [14, p. 80].
Verbal components are the most important modes of communication, because in “typical everyday
communication, they are seen as the keys to the values presented in the messages”. Sounds of
language are considered as one-sided units that have a material physical and acoustic form, but lack
their own semantic content.
Developing the theory of the phoneme, N. S. Trubetskoy has emphasized its active role in
distinguishing, but not in creating meanings. Indeed, not having their own meaning, such as phonemes
as “s” and “t” distinguish the words “yes” and “yet”, acoustically formalizing their meaning [19, p.
72]. Such a minimal unit as a phoneme, which has no semantic meaning of its own, being used in a
fictional prose text, has an additional aesthetic and semantic load of expression, because it performs a
visual and expressive function.
Stress also plays an important role [19, p. 73]. After all, it is known that changing the stress on
different syllables of one word can completely make the meaning of the phrase different in which the
word has been used. By dint of it, the reader can determine the social status of the hero, as well as his
cultural and educational level. Stress also serves as an economical and effective tool of natural self-
characterization of the protagonist, which assists with creating the effect of reliability and authenticity
of the narrative [24, p. 130].
A graphon has the same effect as stress does. The term “graphon” was introduced into the
scientific terminological corpus by V. A. Kukharenko to denote intentional distortion of the spelling
norm in order to reflect the violation of the phonetic norm, which reflects the individual or dialectal
peculiarities of pronunciation [24, p. 135]. In modern linguistic studies, the graphon is also
determined as a phonographic stylistic device, graphic or graphostylistic tool. The synonymous
functioning of the mentioned definitions makes it difficult to identify that linguistic phenomenon due
to the fact that the graphon is considered as a unit of different language levels [14, p. 85]. Graphons
do not only characterize the hero, but also reflect the ironic and author’s attitude to him. They may not
even violate the sound norm, but when they are written, they make the reader’s impression on the
character. F. de Saussure explained that the written word displaces spoken one in our minds [17, p.
150]. That is why all the graphons, in which the normativity of sound prevails over the normativity of
writing, depict a hero with a low culture of speech in the addressees’ minds [17, p. 155].
There is also a graphical representation of graphemes. There are a variation of text types and ways
of graphically presenting a word, such as hyphenation or doubling (tripling) of individual graphemes
in fictional prose text. Well-known linguist B. de Courtenay has noted that visual alternations of
graphemes can be used due to the difference in morphological and semasiological representations of
imagination because of connection with form and meaning. The change of the text type and line
narrowness of the graphemes influences the intonation and logical emphasis and performs the
functions of transmitting the emotional state of the speaker at the time of speech. Hyphenation, as a
rule, serves to depict the strong arousal of the character, italics indicate the intensification and / or
transfer of phrasal stress in the word [13, p. 221].
Since a phoneme under special conditions of its operation has been updated and becomes a carrier
of additional information of the fictional prose text, we can assume that the morpheme, a unit of the
next level, which has not only form but also its own content, makes an even more significant
contribution to the text content [24, p. 135] Morpheme is the main component in word formation, and
for inflectional languages and word change. It is known that the enrichment of the dictionary is
carried out primarily by creating new words from the fund of existing morphemes. Lexical
innovations of a literary text are an important component of the author’s point of view, so they are a
source of language enrichment. The emergence of individual author neologisms (occasionalisms)
indicates not only the development of language, but a change in point of view and ways of reflecting
reality [24, p. 319].
Occasionalism is an unusual, expressively colored lexical word that renames familiar objects and
phenomena. The unusual combination of morphemes in occasionalism attracts the reader’s attention,
because it is a deep fresh thought embodied in a new form. Its occurrence is due to two reasons: 1)
incompleteness of the word-forming paradigm of the word, in which there is no unit with the
necessary morphological and syntactic characteristics and 2) incompleteness of the word-changing
paradigm [16, p. 91]. The functions of proposing an occasional combination of morphemes in a new
word are not limited to the economy of language means and expressiveness. As a bilateral unit, the
morpheme participates in the “morphemes game”, which, like the play on words, is based on
polysemy and homonymy of the units used. Morpheme play is functionally similar to word play,
because it also has a pronounced authorial modality, usually humorously ironic or grotesque-satirical,
and structural reception is carried out, mainly by repeating the actualized unit in it [3, p. 55]. There is
nothing unusual, individual, one-time, temporary in the very structure of occasionalism, in the models
based on which they are built. Their peculiarity is in the unusualness of the lexical compatibility of
morphemes, in its individual character. Every new word comes out of never incompatible morphemes.
Regardless of whether the morpheme will eventually enter national use or remain a one-time,
situationally fixed word, it always occurs for the first time in someone’s individual language, and the
role of literature masters in this process is difficult to overestimate [16, p. 92]. Thus, the actualization
of a language unit can be considered as a fictionally significant fact only in connection with the
performance of its informational-aesthetic function, included in the general artistic perspective of the
work. Morpheme is involved in saturating the text with additional content and modality. Thus, despite
the fact that the morpheme is a bound form and deprived of independent functioning in the language,
it is able to create additional content in the text, performing the functions of nomination (occasional
combination of morphemes) and logical-emotional intensification (morpheme repetition) [4, p. 335].
The lexical level is the next difficult level of the language hierarchy after the phonographic and
morphological ones. The importance of the word for all life and human activity cannot be
overestimated. The word denotes all objects, processes, phenomena that are around us, without the
word communicative activity does not make any sense [8, p. 142].
It is known that data about the frequency of use of lexical units in speech are necessary for various
areas of theoretical and applied linguistics. Frequency dictionaries of all languages, all texts, samples
of various lengths show the same statistics: they are all headed by synsemantic words. Articles,
prepositions, local-temporal adverbs, pronouns fill the first hundred positions of frequency
dictionaries of different languages. They consist of only about 1% of the lexemes, but they cover
almost half of all word usages in the text [15, p. 391]. Words of relative semantics (synsemantic) are
semantically and grammatically insufficient, so they cannot independently fill this or that position in
the sentence structure, but require independent words in the appropriate grammatical form. Such
words, though full-fledged, but with open semantics, need to fill the content of the sentence with
words that depend on them. Thus, synsemantic words together with independent words form
analytical forms of expression of sentence members. Words of different parts of speech can have
semantic incompleteness (synsemantics), but often transitive verbs have this feature [25, p. 114].
A person’s ability to perceive and interpret connections and relationships relates to the cognitive
sphere of activity and requires complex mental operations. Thus, to identify the cognitive basis of the
syntax of the literary text, it is more logical to refer not only to the systematization of syntactic units,
but to identify patterns of syntactic connections and relationships [5, p. 170]. The author’s concept of
the organization of the text space introduces long, complex sentences with various types of lexical and
syntactic connection into the text and encourages the author to choose additional punctuation tools.
Punctuation is an important way of conveying whole sets of authorial meanings and intentions
(stylistic, pragmatic, expressive, etc.), and not just markers of boundaries between segments of
utterances, which in turn are “elements which arise from the division of the text”, the syntactic
organization of which is reflected at the prosodic level [7, p. 97]. However, the text interpretation is
no limited to the identification of actualized elements of the linguistic matter of the works. It provides
only the first stage of the process of high-quality processing of fictional information. Among the
many identified specific cases of actualization, it is necessary to establish their hierarchy, identify the
leading functions, determine the dominant. To do this, we have interpreted a fictional text as a
communicative unit of the highest level of complexity [1, p. 40].
Content-conceptual data informs the reader about individual-author’s understanding of the
relations between the phenomena described by means of content-factual information, understanding
of their causal relations, their significance in the social, economic, political, cultural life of the people
[31–43], including relations between individuals, their complex psychological and aesthetic-cognitive
interaction. Such information is extracted from the whole work and is a creative rethinking of these
relations, facts, events, processes occurring in society and represented by the writer in the imaginary
world created by him. This world closely reflects the objective reality in its actual embodiment [10].
It can be stated that informativeness, which is a textual category, applied to a fictional text should
be marked as a category of heterogeneous multi-channel informativeness, which, despite its
heterogeneity, has a single focus on the disclosure of the concept of the work. This subordination of
local and global, micro- and macro modes and functions of a single task provides their close
relationship and interaction – their system, which is also a categorical feature of the fictional text [9,
p. 139].
The nonverbal component as well as the verbal one is an obligatory component of a multimodal
fictional prose text. Nonverbal components of communication can also act as an independent mode of
information and convey the content of the text in full (iconic components), or as ancillary to verbal
components and add additional semantic nuances to its content (paragraphemic components) [6, p.
574]. Paragraphemic and iconic components are nonverbal modes of a multimodal fictional prose text
[29, p. 37].
Figure 3: Multimodal components of modern fictional prose text
The increase in the number of nonverbal components in the text and the expansion of their
functions in the organization of the information continuum of the text is due to the author’s creative
search for new modes of solving communicative and pragmatic problems. Under the influence of the
oral analogue, the written text becomes a unit of nonlinear perception and understanding of
information. Printed (electronic) text becomes the main in the communicative activity of the addresser
and the addressee. The authors of multimodal fictional prose text use a variety of technical modes to
create an original text, which affects significantly the process of generating text at the internal and
external levels. Changes in value, cognitive and technical factors affect speech and mental processes,
social and speech behavior of native and non-native speakers [2, p. 16].
The Python software product identifies text highlighted by italics, bold, underlining and
strikethrough, which hastens the process of collecting linguistic material in multimodal fiction.
Multimodal fictional prose text uses in its texture a variety of semiotic modes of presenting
information simultaneously. After all, multimodality is closely related to semiotics and interacts with
discourse analysis, functional linguistics and sociolinguistics. Nonverbal components of
communication can act as independent modes and convey the content of the text in full (iconic
components), or as ancillary to verbal components and make additional semantic nuances in its
content (paragraphemic components) [11, 12]. Punctuation dividing a sentence into constituent parts
plays an important role among graphic components in a multimodal fictional prose text. Since the
perception of the text occurs through the visual channel, graphic design is an essential condition for
its adequate perception and understanding. In the multimodal fictional prose text of the postmodern
period, paragraphemic components, on the one hand, appear as markers of intonation, and on the
other – i.e. as modes of simplifying its visual perception.
Conclusions
The modern multimodal novel is characterized by inclusive content, large-scale epic narration and
graceful characters. It consists of symbolic images and extended metaphors which enrich the content
and give special expression to the problems that arise. The synthesis of real and conditional forms
allows modern writers to develop a colorful multilayered content. Postmodern prose is also
characterized by its polyphony, which is a specific feature of the composition, the space-time novels
continuum. The most important role is played by the social and cultural life of the characters. The
multi-genre nature enriches the structure and content of modern multimodal novels. Analyzing the
multimodal understanding of modern novels, it is vital not only to interpret the meaning made up by
different semiotic channels of information, but also to consider what values are created due to the
specific interaction of different channels of information in a particular fragment of text. The modes of
verbal and nonverbal components function not only as separate ways of creating meaning, but
combine and integrate, forming a complex whole that cannot be explained primitively. Thus,
multimodal fictional prose text is a modern phenomenon that combines verbal and nonverbal
components to transmit relevant content through a variety of semiotic channels of information.
Furthermore, the perception of the text occurs only with the correct interpretation of its component
functions.
3. References
[1] E. E. Anisimova, Creolized texts – texts of the XXI century? The interaction of the verbal and
paralinguistic in the text, Voronezh, 1999, 146 p.
[2] I. V. Arnold, Graphic stylistic means. ACM 3 (1973) pp.13–20.
[3] Sh. Bally, Language and life, Editorial URSU, Moskow, 2003.
[4] Y. A. Belchikov, Interaction of functional varieties of language (Contaminated texts), Culture of
Russian speech and communication efficiency, Nauka, Moscow, 1996.
[5] E. Benveniste, Semiology of language, General linguistics, Moscow, 1974.
[6] I. A. Bekhta, Text and linguistics of the text, Visnyk of Lviv, Lviv, 2000, pp. 570–583.
[7] I. A. Bekhta, Discourse of the narrator in English prose, Gramota, Кyiv, 2004.
[8] T. M. Dridze, Text activity in the structure of social communication, Nauka, Moskow, 1984.
[9] M. A. Karp, Interaction of modes of contaminated lexical cohesion in the text of English
multimodal literary fairy tales by Philip Ardagh, ACM 62 (2016) pp. 136–139.
[10] M. A. Karp, Contaminated coherence in dialogues and polylogues of English multimodal literary
tales of Philip Ardagh, ACM 14 (2020) pp. 112–118. http://doi.org/10.32782/tps2663-
4880/2020. 14-1.20.
[11] M. A. Karp, T. V. Yaremchuk, Functions of the verbal component in the multimodal fictional
prose text of Jonathan Safran Foyer’s novel EXTREMELY LOUD AND INCREDIBLY
CLOSE, ACM 11 (2020).
[12] M. A. Karp, T. V. Yaremchuk, Functions of the iconic component in the multimodal fictional
prose text of Jonat Safran Foyer’s novel EXTREMELY LOUD AND INCREDIBLY CLOSE,
ACM 12 (2020).
[13] E. V. Kulikova, Graphic means as stylistic methods of fictional text, ACM 23 (2001).
[14] V. A. Kukharenko, Text interpretation, Education, Moscow, 1998.
[15] O. Potebnia, Thought and language, Erased, Kyiv, 2012.
[16] G. Ya. Solganik, From words to text, Nauka, Moscow, 1993.
[17] F. de Saussure, General linguistics course, Works on linguistics, Moscow, 1977.
[18] O. V. Spodaryk, British multisemiotic fictional text of the beginning of the XXI century:
semantic, structural and functional dimensions, Chernivtsi, 2015.
[19] N. S. Trubetskoy, Classification of oppositions, Fundamentals of phonology, Aspekt Press,
Moscow, 2000.
[20] M. A. K. Halliday, Cohesion in English, Research on text theory, Moscow, 1979.
[21] R. O. Jacobson, Language in relation to other communication systems, Selected works,
Moscow, 1985.
[22] A. Baldry, P. Thibault, Multimodal transcription and text analysis, A multimedia toolkit and
coursebook, Equinox, London, 2006.
[23] H. Bussmann, Routledge dictionary of language and linguistics, Routledge, London, 2006.
[24] D. Crystal, A dictionary of linguistics and phonetics, Blackwell Publishing Ltd, 2008.
[25] M. A. K. Halliday, An introduction to functional grammar, Edward Arnold, London, 1990.
[26] M. A. K. Halliday, Explorations in the functions of language, Akademia Press, London, 1973.
[27] M. A. K. Halliday, Language structure and language, New horizons in linguistics, Penguin,
London, 1970.
[28] G. Kress, T. van Leeuwen, Reading images, The grammar of visual design, Routledge, London,
1996.
[29] G. Kress, Multimodality: A social semiotic approach to contemporary communication,
Routledge, London, 2010.
[30] N. Nørgaard, Multimodality and the literary text: Making sense of Safran Foer’s Extremely
Loud and Incredibly Close’ in New Perspectives on Narrative and Multimodaliy, London & New
York, 2009.
[31] V. Pasichnyk, V.Lytvyn, N.Kunanets, R. Vovnyanka, Y. Bolyubash, A. Rzheuskyi, Ontological
approach in the formation of effective pipeline operation procedures, in: 13th International
Scientific and Technical Conference on Computer Sciences and Information Technologies,
CSIT, 2018, pp. 80–83.
[32] A. Rzheuskyi, N. Kunanets, V. Kut, Methodology of research the library information services:
the case of USA university libraries. Advances in Intelligent Systems and Computing 689 (2018)
450–460. doi:10.1007/978-3-319-70581-1_32.
[33] M. Odrekhivskyy, V. Pasichnyk, A. Rzheuskyi, V. Andrunyk, M. Nazaruk, O. Kunanets, D.
Tabachyshyn, Problems of the intelligent virtual learning environment development. CEUR
Workshop Proceedings 2386 (2019) 359–369.
[34] V. Tomashevskyi, A. Yatsyshyn, V. Pasichnyk, N. Kunanets, A. Rzheuskyi, Data Warhouses of
Hybrid Type: Features of Construction. Advances in Intelligent Systems and Computing book
series 938 (2019) 325–334.
[35] R. Kaminskyi, N. Kunanets, V. Pasichnyk, A. Rzheuskyi, A. Khudyi, Recovery gaps in
experimental data. CEUR Workshop Proceedings 2136 (2018) 108–118.
[36] A. Rzheuskyi, H. Matsuik, N. Veretennikova, R. Vaskiv, Selective Dissemination of Information
– Technology of Information Support of Scientific Research. Advances in Intelligent Systems
and Computing 871 (2019) 235–245.
[37] A. Shakhov, V. Piterska, O. Sherstiuk, O. Rossomakha, A. Rzheuskyi, Management of the
technical system operation based on forecasting its “aging”. CEUR Workshop Proceedings 2565
(2020) 130–141.
[38] R. Kaminskyi, N. Kunanets, A. Rzheuskyi, A. Khudyi, Methods of statistical research for
information managers, in: Proceedings of the 13th International Scientific and Technical
Conference on Computer Sciences and Information Technologies, CSIT 2018, 2018, pp. 127–
131.
[39] A. Rzheuskiy, N. Veretennikova, N. Kunanets, V. Kut, The information support of virtual
research teams by means of cloud managers. International Journal of Intelligent Systems and
Applications 10(2) (2018) 37–46.
[40] E. Vasilevskis, I. Dubyak, T. Basyuk, V. Pasichnyk, A. Rzheuskyi, Mobile application for
preliminary diagnosis of diseases. CEUR Workshop Proceedings 2255 (2018) 275–286.
[41] V. Piterska, S. Rudenko, A. Shakhov, Development of the method of formation of the
architecture of the innovation program in the system Univers -State-Business. International
Journal of Engineering and Technology(UAE) 7(4.3 Special Issue 3) (2018) 232–239.
[42] V. Piterska, A. Shakhov, Development of the methodological proposals for the use of innovative
risk-based mechanism in transport system. International Journal of Engineering and
Technology(UAE) 7(4.3 Special Issue 3) (2018) 257–261.
[43] M. Odrekhivskyy, N. Kunanets, V. Pasichnyk, A. Rzheuskyi, D. Tabachishin, Information-
analytical support for the processes of formation of smart sociopolis of Truskavets. CEUR
Workshop Proceedings 2393 (2019) 241–256.