The article analyzes the intelligibility of phonemes at various mid-frequency intervals. As part of the work, 6 vowel phonemes were considered, a table of their intelligibility was formed and analyzed. The correctness of the work was assessed by listening to the changed sound track by 7 persons. The results obtained allow us to identify parts in the spectrum that most influencing the intelligibility of phonemes. That parts can be used to assess the quality of speech and the intelligibility of phonemes during the rehabilitation of patients after surgical treatment of the organs of the speech-forming tract.
To date, automatic speech recognition systems have achieved significant results, investing and
promoting commercial applications in this area is beneficial. So, according to BCC Research, by 2021
the world market for speech recognition technologies will be estimated at $ 184.9 billion [
Continuous speech recognition [
Another important area where speech recognition can be used is the assessment of its quality, in
particular, intelligibility. To assess the quality, standard approaches [
One of the important aspects in speech recognition is data preparation. Analysis methods can be resource-intensive, therefore, preparing an optimal data set in terms of volume is an urgent task. In terms of optimizing the size of the parameters, the potential reduction in the frequency analysis ranges is important. Filtering out areas that do not affect legibility can have a significant impact on system performance or reduce the amount of resources it consumes.
It is also important that the application of many existing methods for assessing intelligibility
requires the participation of experts and, as a result, does not claim to be completely objective.
However, such methods claim to be objective by increasing the number of experts, for example, up to
five people [
Based on these considerations, preliminary data analysis with the aim of forming the most informative features for further analysis using machine learning methods is an urgent task.
In order to increase the capabilities of continuous speech systems, it is necessary to consider sounds separately. In this work, the features of the behavior of phonemes at various intervals of midfrequencies are considered and analyzed.
The basic unit of the phonetic level of the language is the phoneme. The concept of a phoneme is
associated with the development of understanding of language as an integral system. Professor of
Kazan University I. A. Baudouin de Courtenay, who was the first to develop the concept of a
phoneme, emphasized that the allocation of a phoneme is possible only when the entire system of
phonemes of a given language is taken into account [
The phoneme is the minimal meaningful unit of the language, which does not independently have lexical or grammatical meaning, but serves to distinguish morphemes and words.
The phoneme as an abstract unit of language corresponds to the sound of speech as a concrete unit.
The spectrum of speech sound can be decomposed into tone (periodic) and noise (non-periodic)
components. Tone sounds are formed with the participation of the vocal cords, noise sounds - by
obstacles in the oral cavity. By the presence of these components, the first classification of speech
sounds can be made:
Vowels – tone
Voiceless consonants – noise
Sonorous consonants - tones with a slight admixture of noise
Voiced consonants - noise with tone participation [
Differential features of phonemes are associated with the difference in acoustic features of sounds, which, in turn, is associated with differences in their articulation, that is, with a difference in the work of the speech organs. Voicedness - the presence of not only noise in the sound, but also the tone created by the work of the vocal cords; softness - a large pitch of sound caused by a change in the shape of the oral cavity as a result of additional articulation - the rise of the middle part of the back of the tongue to the hard palate.
But for the selection of phonemes, it is not the articulatory and acoustic aspects of these signs themselves that are important, but their opposition, their use to distinguish other linguistic units. The vowel sounds [a], [o], [i] can be pronounced in Russian with different durations (compare the extension of vowels in words when expressing surprise, doubt, indignation, etc.: [ta: m?], [ kn'iga?], [vo: n!] and under.), but the duration of pronunciation in Russian is not used to distinguish between words and forms of words, and therefore, the difference between sounds [o] and [o:], [i] and [i:] do not develop into phonemic differences.
Phonemes that differ in only one differential feature are called paired. Paired in Russian are the phonemes [b] and [b'], [b] and [p], [d] and [d'], [d] and [t] and so on. For example, the phonemes [ts] and [ch] are unpaired, since there is no phoneme that would differ from [ts] or from [ch] just one feature.
Different languages have different types of syllables. The types of syllables differ according to the
ratio of the syllable (G) and non-syllable (S) element. When taking into account the end of a syllable,
open - the syllable ends with a syllable element (SG) - and closed - the syllable ends with a
nonsyllable element (GS) - syllables. When taking into account the beginning of a syllable, they
distinguish between covered (first sound of a non-syllable) and naked (first sound of a syllable)
syllables. In the word [o\kno], both syllables are open, but the first is open and the second is covered,
in the word [go\rod] both syllables are covered, but the first is open and the second is closed [
For the Russian language, open syllables are more characteristic, consisting of a consonant and a vowel, they make up more than half of all syllables found in speech. Based on this, we can conclude that the analysis of vowel phonemes is an important component of speech recognition and assessment of its quality.
In this work, audio recordings are taken from the base of vowel phonemes of the male and female speaker, vowel phonemes are recorded, and also taken from the database of audio recordings of the syllables of the female and male speaker. Audio files in Russian were used for the analysis.
Number of syllables (recordings) from a speaker: 50.
Vowel phonemes: [a], [i], [o], [u], [ɨ], [e].
Before use, audio files were converted to wav format, 16 bit, mono.
The number of auditors is 7 people.
Total number of assessments received from auditors: 350.
Since one of the goals in the work is to demonstrate the problems in assessing intelligibility by
expert methods even on small amounts of data, it was not the goal to form a large set intended solely
for the application of automated analysis methods based on machine learning methods. It was
necessary to identify and evaluate the problems associated precisely with the accuracy and objectivity
of the obtained assessments of intelligibility when using standard expert assessments [
The Butterworth filter [
The following division of the total frequency range of 200-3000 Hz into sub-ranges was used, presented in tables 1-3.
This division is associated with the distribution of the resonant frequencies of the speech-forming
tract (formant) for the first formant (table 1) and the second formant of various phonemes (tables 2
and 3) [
For each of the subranges, the intelligibility was assessed as the proportion of correctly defined phonemes averaged over all speakers.
Further, on the basis of the obtained intelligibility values, the most important parts of the spectrum of vowel phonemes influencing the intelligibility of vowels were found. 4.
Let's consider the order of research using the example of the phoneme "a". 1. Select the phonemes of interest 2. We carry out filtering to select the frequency range of interest 3. Save the resulting file 4. Submitting to the experts for assessing intelligibility
Signal spectra for the ranges 200-600 Hz, 200-400 Hz, 300-500 Hz and 400-600 Hz are shown in Figures 1 a-d.
The results of the assessment of intelligibility for the given fragments for the first auditor: 1, 1, 0, 0.
Similar assessments were carried out for all auditors, phonemes and spectrum regions. The final results for assessing intelligibility for isolated phonemes are presented in Tables 4-6.
Similar characteristics for male and female voices are presented in Table 7, the total is averaged over all vowels. 0 0 0 0 0 1 1 1 0 0
Once the intelligibility scores have been obtained, you can analyze and compare them.
It can be noted that in the interval from 1800 to 3000 Hz, intelligibility is not preserved for all vowel phonemes. As an exception, the phoneme [a] and [i] can be distinguished, their ranges are close to high frequencies, but to maintain intelligibility, a wide band is required at intervals of 400-800 Hz. You can also see that the intelligibility of the male voice is lower in the frequency range than the female, which is most likely due to the lower value of the pitch frequency.
It can also be seen that some of the listeners noted the intelligibility of the phoneme [a] in the range from 400 to 600 Hz and in the band from 600 to 800 Hz, therefore, you need to look at the frequency range from 500 to 700 Hz. Additional analysis of this range confirmed the intelligibility within its limits equal to 1.
Phoneme [a] is partially legible at 1000-1800 Hz and 1000-1400 Hz, which means that for legibility it may be necessary to partially expand the range towards 200-1000 Hz. As a result, with the extended range, it turned out that intelligibility was preserved in the range 800-1400 Hz, while the frequencies in the range significance for 400- 600 Hz and 600-800 Hz cannot be denied.
Phoneme [i], the frequency range from 700 to 900 Hz, 14% of listeners noted that intelligibility was preserved, since in the range from 800 to 1000 Hz, intelligibility is preserved, it is possible to distinguish that the range partially coincides, but the main band lies higher, and this assessment is related to individual differences of the listener.
Next, consider the phoneme [i], as with the phoneme [a], we will shift the frequency range and evaluate the results. intelligibility appeared in the 1400 to 2200 Hz range, but as the range decreases, the intelligibility begins to fade, but these ranges significance cannot be denied.
For phoneme [o], the range was extended in the same way as for phoneme [a]. It can be concluded that a wider range is required for intelligibility and the data is due to the individual listener, but this range significance cannot be denied.
In the phoneme [u] in the range 300-500Hz, 14% of the listeners emphasized intelligibility, this is due to the individual differences of the listener and this range significance cannot be denied.
After analyzing the results in syllables, a clear difference can be distinguished in that a wider band is required to preserve audibility, and audibility is also preserved not for a single phoneme, but for the syllable as a whole. As a feature, the intelligibility of the syllables stands out in the aggregate, during the listening, the falling out syllables did not stand out, as a result, the intelligibility or absence is preserved for the entire audio file as a whole.
An intelligibility table was compiled for all midrange syllables. The data can then be used to develop a module for assessing the quality and intelligibility of speech.
In the course of the study, it can be concluded that the frequency ranges corresponding to the most
informative set of features can be compared to the frequencies of the formants of sounds and the
frequency of the main tone of the speaker. Indirectly, this dependence depends on the gender of the
speaker through its influence on the frequency of the main tone [
In addition, the studies carried out have clearly confirmed that the use of existing expert methods
for assessing intelligibility can introduce significant contradictions due to differences in the
perception of messages by five (the recommended number [
This research was funded by a grant from the Russian Science Foundation (project 16-15-00038). 8. References