The article provides an attempt at systematization of the elements of oral discourse which are not related to the text content but are nonetheless very frequent in everyday speech and thus essential for its understanding and decoding. Nonverbal elements can be tracked almost in any type of spoken speech or any given speaker. Therefore it is essential to have a comprehensive classification which will enable researchers to deal with spoken speech data with more precision. Such elements include some filled hesitation pauses such as [ə:], [ə:m], [i:], [n], etc., nonverbal vocalizations like clicking, lip-smacking and squelching, as well as a number of other paralinguistic elements (voice qualifications such as laughing, sighing, coughing and so on). The aim or the paper is to list various nonverbal elements in The Speech Corpus of the Russian Language (amounting to 1280 hours of recorded everyday Russian speech of more than 250 respondents and about 1000 of their interlocutors) and categorize them with regard to their pragmatic meaning. Nonverbal vocalizations usually tend to fill the hesitation pauses marking the so-called points of failure. Moreover, they often help to structure a text being produced and sometimes perform several functions simultaneously. While being hesitative, can also perform search functions (when a speaker searches his mind for a word, an expression or an idea to continue or complete an utterance), be a reflexive marker or as a discursive marker of the speech start or finale.
It has been widely acknowledged that in contrast with written discourse, spoken
speech has its own rules and therefore requires special research methods and
approaches. In order to help describe and analyze contemporary Russian speech, three
key elements have been drawn up [
Verbal elements are in the core of the semantic dimension of a text; they carry the
principal meaning of a message. Roughly verbal elements are characterized by high
frequency and high repetition; they help structure the text without actually being
connected to its meaning. They are auxiliary parts of speech and parenthetical words.
Moreover, to this category belong pragmatic markers, for example, verbal hesitatives
of search (“kak ego” ‘whatshisname’). The research of Russian pragmatic elements, if
not sufficient, is definitely striving at the moment; we can consider the studies of
K. L. Kiseleva’s and D. Paillard’s works [
Nonverbal elements of speech stand out in every utterance because they are rather frequent, yet they do not seem to bear any significance with regard to an utterance meaning. Apparently, being highly repetitive, they can structure and even pace a text without actually being of textual nature. These elements include hesitation pauses as a major part of spoken discourse.
While describing nonverbal communication in English, which usually implies
visual information like gestures from face, eyes, hands and other body parts, D. Crystal
[
These elements have also been found rather frequent in everyday speech; however,
their research in Russian speech has been devastatingly scarce. These non-verbal
elements of the utterance are considered to be a type of speech malfunctions
disrupting the smooth deployment of the speech (disfluencies) [
Non-verbal elements of speech, first and foremost, are hesitation pauses filled with
non-phonemic sounds, or vocalizations. Pauses are considered to be an essential
criterion for fluency rating and speech rate measurement. As a rule, pauses in speech are
categorized into filled and unfilled, the former being hesitation particles like [ə:] or
[ə:m] and the latter a simple silence. Filled pauses are an important indicator of
speech fluency and therefore are widely investigated in studies dedicated to second
language acquisition and mastering [
It is the assumption that, in comparison to native speech, in non-native language
the number of hesitations increases, which enforces the effect of slowing down and
reduced fluency. However, it has been observed that "filled pauses" rarely occur in
read speech [
Clicks are usually described as phoneme realizations in some African languages [
Another recent discovery suggests that clicks are, presumably unintentionally, used
as discourse markers indexing a new sequence in a conversation or before a word
search. For example, J. Trouvain and Z. Malizs [
In Russian research clicks have rarely been identified and studied; however, some
[
Physiological reflexes such as chewing noises, hiccup, coughing, yawning etc. are not
usually considered communicative because they are not always under control of the
speaker. However, some deliberate vegetative sounds (such as clearing the throat as
indicating one’s presence) can have pragmatic meaning and thus deserve further
investigation [
Affect bursts [
It is generally believed that nonverbal vocalizations occur more often in
conversational speech than in monologues, reading at loud or other forms of controlled
speaking. An analysis of six corpora of conversational speech [
In addition to nonverbal vocalizations which can be investigated in several
languages, there are those less widely acknowledged, e.g. lip-smack which is consistent
with the Chinese language. It is a sound generated by pressing lips together and then
opening them quickly, and it is considered to be a typical background event in
Chinese spontaneous speech [
To summarize, we can see that nonverbal elements are very common in spontaneous speech. When conducting a thorough multi-level analysis of verbal spoken speech, one must detect and categorize its inherent nonverbal elements to help investigate and process more significant textual parts of any utterance. The current study is a part of ongoing research into pragmatics of spoken Russian, and based on this we now formulate the following research questions: 1) Which non-verbal elements can be found in everyday Russian speech? 2) How can we categorize them? 3
This study is conducted on the two modules of the Corpus of the Russian language:
the corpus of Russian everyday speech “One Day of Speech” (the ORD corpus,
containing mostly dialogic speech) [
The ORD corpus captures natural speech of native speakers (residents of St. Petersburg who speak Russian as their native language) and contains mostly everyday dialogues and polylogues, recorded using the method of continuous daily speech monitoring and recording. Each respondent provided about 8-14 hours of speech recordings which were then converted to the format of the corpus: PCM, 22050Hz, 16 bit, mono, while the original recordings had been stored in the archive. Next, the recordings were segmented into the so-called macroepisodes, in other words, fragments homogeneous in their communication settings which may include the place of communication, its settings, social roles of speakers or the activity they engage in. This segmentation was performed manually by qualified linguists who listen to the recordings and mark the boundaries between episodes.
The phonetic quality of each macroepisode is evaluated and measured in a 4-grade
scale: 1 – the best quality, suitable for precise phonetic/prosody analysis, 2 – rather
good quality, which is partially suitable for phonetic analysis, 3 – noisy recordings of
intermediate and low quality, which are not suitable for phonetic analysis but are
suitable enough for other aspects of research, and 4 – unintelligible conversations or
remarks in extreme noise, which could not be understood without noise reduction
techniques [
All data has been manually transcribed and later verified in ELAN [
Currently the corpus comprises 1250 hours of sound recordings, collected from 128 respondents and more than 1000 of their interlocutors, representing different social groups of St Petersburg, Russia, 2800 macroepisodes of communications, and more than 1 mln word usages in transcripts.
SAT, on the other hand, contains a less natural experimental speech. These are the monologues recorded from native speakers of different professional groups: doctors, lawyers, computer scientists, teachers of language and philosophy, various groups of students, incl. those majoring in language, and so on. SAT recordings are categorized into a series of typical communicative scenarios of everyday communication: reading, retelling, description of the image, story-telling. In addition to the speech of native Russian speakers, SAT also includes several blocks of L2 Russian speech by nonnative speakers: American, French, Chinese, and Dutch. At the moment, the collection includes data obtained from 153 speakers and comprises 772 monologue texts, with total duration of 30 hours.
In brief, all data in the corpora is presented in both audio files and transcripts. An annotated ELAN file is presented in Fig. 1.
It can be seen that there are certain symbols used in transcripts to mark non-verbal phenomena (*C, *B and others). Most common symbols include “*П” for a hesitation pause, “/” for a short utterance pause and “//” for a long pause marking the end of an utterance. Other symbols are introduced in their respective sections. All words and utterances are given in orthographic writing.
For the current classification study we explored both types of records and identified the phenomena we thought to be of non-verbal nature. Then we analyzed the phenomena and classified them into categories. It should be mentioned that so far the analysis is of qualitative nature rather than quantitative, principally because we aimed to create a classification to be proved or disproved in further research into spontaneous Russian speech.
4.1
Being the pioneers of comprehensive descriptions of Russian nonverbal vocalizations, we are faced with a series of debatable issues.
Firstly, we would aspire to compare and contrast our classification to those already existing in describing other languages, mainly English. So, one is expected to come up with an inventory similar or of the same nature, operating more or less similar terms and definitions. However – and here comes our second stumbling stone – there are abovementioned Russian studies of some, if not all of them, nonverbal elements in spoken Russian, and as native researchers we would not want to digress too far from our venerable colleagues.
As a result of our investigation, we have come up with a working theory for the typology of nonverbal elements in spoken Russian speech. In the corpus recordings managed to track the following elements: Hesitation pauses (filled and unfilled); Clicks; Lip-smacks; Noisy air intakes; Voice qualifications, or affect bursts.
This inventory serves as an exploratory one which is liable to undergo some alternations or refinements in the process of its validation on perhaps expanded speech material.
Hesitation pauses. Both types of hesitation pauses, filled and unfilled, can be
found in the corpora, and they are rather frequent. Given that some subcorpora have
been described in previous research, we can preview some quantitative data. For
instance, in the SAT reading recordings (subgroup STU) there are 323 hesitation pauses
[
There are different non-phonemic sounds that can fill a pause, predominantly [ə:] or [ə:m], [a:], [a:m], [m:]. In the Russian L2 speech of native Chinese speakers it was possible to trace sounds such as [y], [yn], [n:]. An example of a hesitation pause is provided below (see Fig. 2). The main function of filled pauses is hesitational search, either for a specific lexical unit or signal general speech formulation difficulty. More often than not, this search function seems to be accompanied by others. Let us consider some examples: (1) i neskolko [m:] dvorovykh malchishek s treshchotkami (SAT, reading); (2) Grigorij_Ivanovich [ə:] Muromskij [ə:].
Thus, in these examples, the speaker seems to hesitate before an archaic word
uncommon for contemporary speech “dvorovyj” 'house serf' (1) and surname
“Muromskij”. A previous study of this corpus data on lexical and syntactic level have
previously suggested that there are markers of speech non-triviality which signal
introducing some extraordinary, non-so-common verbal units, and they are often
accompanied by hesitation pauses [
The vocalizations often come together with other pragmatic markers, such as reflexive markers, markers of hesitation, or discursive markers. Thus, we may assume apparent polyfunctionality of vocalizations in oral discourse, with the hesitativesearch character of almost all such elements as a given.
Clicks. As we already mentioned, clicks are not often specified in Russian research into spontaneous speech. In the current study, it was possible to locate clicking, marked as *Ц in the corpus transcripts (see Fig. 3), in spoken Russian material, both in native and non-native speech.
However, more often than not clicks would be attributed to Chinese speakers. In most cases, clicks would definitely be of hesitative nature, and their primary function is word search, be it successful or not quite: (3) “nuzhno *Ц / za... / zanimatsya; ne ochen’ [ə:] nravits’a(:)[əm:] *П *Ц khodit’ v magaziny” (“I have to study, I don’t really like going shopping”).
Similar to hesitation pauses, in clicks search function is also combined with the discursive start function: the speaker is found clicking at the start another fragment of his monologue. Again, there is some polyfunctionality of the non-verbal elements in oral communication.
Lip-smacks. Lip-smacks are marked as “mp” in our speech corpora findings and typescripts, mainly because of onomatopoeic reasons. These elements seem not very common, yet far from non-existent to be disqualified. It seems that the lip-smacks in spontaneous Russian, as well as all other types of hesitation phenomena, gives the speaker a short break for decision to continue speech or choosing the right word or expression and thus has a search function. Other markers of hesitation have been spotted in the vicinity: nonverbal sounds, prolongation of sounds, word breaks, parasite words and physical pauses, which further enhance their hesitational character.
Noisy air intakes. During this nonverbal vocalization a speaker draws in the air
not through their nose, as it usually happens (including a situation when a deep breath
is a hesitation pause by itself), but through the mouth, with the tip of the tongue at the
front teeth, and between the lateral parts of the tongue and lateral teeth there is a gap
through which the air passes. To an untrained ear it sounds like a noisy air intake. In
some studies, it has been called squelching [
Voice qualifications. There are several types of affect burns recorded and marked in the corpora typescripts, e.g. laughter (see Fig. 4), coughing, yawning, tutting, sneezing, etc. In Russian studies these are often called paralinguistic phenomena. Nevertheless, it seems that the nature of clicks, lip-smacks and noisy air intake would also attribute them as paralinguistic elements, which, however, do not carry much emotional significance. Nonverbal elements of oral speech – in general, as a class of elements, and each separately – deserve a special functional description. But all our examples demonstrate their obvious hesitative nature, and also some polyfunctionality. For instance, clicks, like filled hesitation pauses, in addition to the search function may have the discursive start function. This fact may urge us to review the classification of nonverbal elements in the domain of pragmatics and thus consolidate some elements with regard mainly to their pragmatic function and not their phonetic execution.
At this point we may speak of three principal pragmatic functions: hesitation, search and reflection (often resulting in hypercorrection). 5
Our research shows that there are nonverbal elements in various types of oral discourse, in both monologues and polylogues. On the one hand, these elements do not claim to be significant, or verbal, and surely cannot be described as verbal. On the other hand, they have a definite pragmatic meaning and often help the speaker structure the speech he/she produces.
There are various approaches to categorization of nonverbal elements in spoken Russian speech, however, one cannot deny that these elements must be included in contemporary speech research, given their prolificacy.
The main function of nonverbal vocalizations we have found to be hesitative search, which is often intensified or modified by others: the functions of a discursive marker (start or final), a reflexive or a «non-trivial» marker. Corpus approach to the analysis of oral speech allows not only to identify all such «non-verbal» elements, but also to systematize them.
The findings may be used for many applied purposes: from teaching Russian in a
foreign language audience to automatic speech recognition and linguistic expertise.
Our study has been closely linked with fellow researchers’ work into prosody and
pragmatics, all of us striving to combine prosodic information with pragmatic
annotation of communicative acts presented in the corpora. Further acoustical analysis of our
identified categories of all non-verbal material, which is extremely common in
spontaneous Russian speech, will allow for more precise automatic speech processing.
This research, given its pragmatic aspect, is especially significant with regard to filled
pauses recognition, as it has been observed that ASR systems tend to confuse filled
pauses and backchannels, a functional distinction that humans need to be very good at
pragmatically [
The presented research was supported by the Russian Science Foundation, project #18-18-00242 “Pragmatic Markers in Russian Everyday Speech”.