It is possible to improve the quality of speech recognition in noisy environments adapting the reconstruction algorithm for the recognized word to the certain peculiarities of usage and application. The article describes an approach to reconstruct the phonetic composition of a recognized word using lexical ontology. The lexical ontology contains lexical links among the words of the subject domain and their phonetic composition in terms of the SAMPA+ for the Russian language.
Continuous speech recognition is a complex iterative process, based on the sequential division of the total acoustic signal into words, and then the words into phonemes. At the same time, many factors (e.g., noise) significantly complicate the recognition and reduce its accuracy. Thus, after preliminary speech signal processing many algorithms result in a matrix consisting of a phoneme set with their corresponding detection probability in the recognized word.
However, the task of recognizing and converting Russian continuous speech into a text is an urgent problem. It should be solved in different subject domains: medicine, litigation, automated detection of extremist materials, etc.
At the same time, a special set of the most frequently used terms corresponds to each subject domain. The a priori sampling of the basic terms in the analyzed subject domain helps to increase the probability accuracy while determining the patterns of certain phonetic combinations in the process of speech recognition.
One of the ways to adapt speech recognition algorithms to the corresponding subject domains is to apply lexical ontologies of subject domains.
For the initial extraction of speech units from the recognized speech signal, the algorithm of the phoneme-by-phoneme recognition of the Russian words is used. The initially detected speech units will be used to form word groups taking into account both subject domains and the analysis results of a certain lexical ontology.
For the phonemic recognition of the Russian words from an unlimited dictionary, we use the
following algorithm: a speech segment is divided into constituents [
Then, the speech signal of each speech unit is successively converted into an autocorrelation portrait in the following way. Let ( ) be the i-th readout of a digital speech signal; then ( + ) is a readout spaced readouts apart ( ). Dependency factor of these readouts is expressed by a sample correlation coefficient:
( ) = [ ( ), ( + )] = cov[ ( ), ( + )] = 1 ∑ ( ) ( + ) − [1 ∑
=1
cov[ ( ), ( + )] √1 ∑ =1 2( )− ( )√ 2 1 ∑
2 =1 2( + )− ( + ) =1 ( )] 1 ∑
=1 ( + ), where is a number of readouts in the interval, in which the dependency is sought; cov[ ( ), ( + )] is the sample covariance ( ) and ( + ) when = 1.. ; ( ) is a sample mean ( ) when = 1.. ; ( + ) is a sample mean ( + ) when = 1.. . Function determined by the sample correlation coefficient using (1) is an autocorrelation function of a signal. While its calculation we perform the transformation of a speech signal ( ) = 1.. ( is the number of readouts in a speech signal) into a two-dimensional image. For this purpose, ( ) is divided into intervals including < readouts. Then, using equation (1) we generate image lines: (1) (2) ( ), (
+ ) ==11,.. ,2 ,....
( , ) =
The two-dimensional image X (j, k) obtained from (2), where i is the line number, and k is the column number, is the autocorrelation portrait (ACP) of a speech signal s (i) dimensioned ×
Model speech units with the same parameter value N are transformed into ACPs. These speech
units are arranged from the examples of SAMPA + phonetic alphabet. Thus, it is possible to determine
to what extent the ACP speech unit corresponds to the model ACP. As a result, the speech unit is
considered to correspond to the model with the utmost similarity. The similarity of the ACP is
determined by calculating the Euclidean distance between the ACP lines. The position of each line is
determined in dynamics [
Ontology is a system consisting of many concepts, their definitions and axioms, which are necessary
to limit the interpretation and use the concepts [
OWL (Ontology Web Language) is the Semantic Web language designed to describe classes and their interrelations. At the heart of the language lies the representation of reality in the “objectproperty” data model. OWL is a reformulation of the descriptive logic using XML syntax.
Subject domain ontology is a collection of RDF-triples: subject-predicate-object. In this research
OWL-ontology was used to solve the problem under consideration [
A special type of ontology is lexical (or linguistic). Its distinctive property is to use (lexicalized) concepts (words) together with their linguistic properties in one resource. The main source of concepts in such ontologies are the values of linguistic units. They are also distinguished by a set of relationships,
which usually characterize linguistic elements: such as synonymy, hyponymy,
meronymy, etc. [
To reconstruct the phonetic composition of the recognized word, the elements of phonetic
alphabets, establishing letter-sound correspondence, were included in the lexical ontology structure.
The most widely-used phonetic alphabets are the International Phonetic Alphabet (IPA) and
XSAMPA (as well as SAMPA + modification, including transcriptions of the Russian language) [
The formal model of a certain subject domain lexical ontology may look as follows: O
A, C Ai , P IPA , P SAMPAi , R Ai , i 1, т, where m is the number of subject domains covered by ontology; = subject domains covered by ontology; = { 1 , 2 , , } is a set of terms within the i-th subject { 1, 2, , } is the number of domain; = is a set of phonemes peculiar to the Russian language according to IPA; + = is set of phonemes peculiar to the Russian language according to SAMPA+. This alphabet consists of 89 phonemes.
is a set of ontology ratios within i-th subject domain:
R Ai R Ai , RCAiP , R Ai , RPA1iP2 ,
C P where is a set of links, which form the hierarchy of ontology terms within i-th subject domain; is a set of object properties and data type properties, which determine the relationship between the elements of , PSAMPA+ sets, and also the corresponding properties of these objects.
RpAi {hasIPA, hasSampaPlus, hasExample...}
is a set of relations determining the links among the set objects and PIPA, PSAMPA+ within i-th subject domain. The given property determines whether the phoneme belongs to the phonetic representation of the corresponding term.
1 2 is a set of links determining the probability that the phoneme P2 follows the phoneme P1 within the terms of the i-th subject domain. This value depends both on the frequency of the term in the texts of the i-th subject domain and on the phoneme sequence in the phonetic representation of a certain RPA1iP2 CAi j,k , (3) is the grade of membership of the term c to the subject domain Ai; , is the probability that the phoneme k follows the phoneme j within the terminological frames of the subject domain Ai.
The values of these relations are derived from statistical analysis of large text corpus on a given
subject domain and from phonetic analysis of each marked term. The task of terminology extraction is
solved with the help of semantic algorithms, i.e. thesaurus-based and nested link algorithms [
The thesaurus-based algorithm for terminology extraction from a set of words, which belong to a certain text, using the OWL-ontology, calculates the degree of semantic proximity of input wordgroups to the terms of the subject domain. This algorithm chooses from the set of incoming words/ word groups only those terms and expressions, which belong to the given subject domain.
The degree of semantic proximity of the input word-group to the subject domain kont can belong to the interval from 0 to 1: the closer is the obtained value to 1, the greater is the possibility that this word/ word group is a term.
The thesaurus-based algorithm suggests a direct search for input word lemmas and their combinations among the terms defined in a certain ontology. For this purpose, for every type of ontology, it is necessary to define the property “containsLemma”, which has a line value obtained by lemmatizing (reducing to the initial form) the object name with the help of Mystem (Yandex product). The lemmatization is carried out according to certain morphological peculiarities of the term.
The thesaurus-based algorithm consists of the following stages:
evaluation of the degree of proximity of the input word / word group to each ontology object; search for the OWL-ontology core object, which is most closely associated with the input word / word group.
The reference core object of the expandable ontology, used in the further analysis, has a degree of proximity to the input word / word group, which is calculated by the following formula: m n kt max ( i ) , i1 pi where m is the number of all objects of the OWL-ontology core; pi is the number of words in the lemma of the reference OWL-ontology core object; ni is the number of words from the input word group lemma, which are found in the lemma of the OWL ontology core object.
The general scheme to evaluate the degree of proximity of the word groups to the terms of the subject domain is shown in Figure 2.
In this case, the word order in the group of words in the reference object should not change.
If several different ontology objects have the same value of the coefficient kt,, then the object that corresponds to the maximum ni. will be considered as a reference one. If there are several maximum objects, then all of them will be considered reference ones and the analysis according to ontological criterion will be carried out for each of them.
The ontology structure of the analyzed subject domain assumes that each of its objects has a Datatype Property “a Term”, which is of a logical type. This property is auxiliary and is determined by the expert who distinguishes to what extent this ontology object is peculiar for a certain subject domain.
Thus, the terminology degree of the input word group is calculated in the following way: kOnt ckt 1 , where kt is a value obtained during the first stage of the algorithm execution; c is the number of ratios between the reference object and the nearest ontology object that has a Data Property “a Term”=true (if this property is true for the reference object, then c=0). The search scheme is shown in Figure 3.
If for the reference object “a Term”=false, and in addition the object has no links with other ontology objects, or all related objects are also distinguished as “false”, then it is necessary to find another reference object for the given word / word group and the evaluate it. In a similar situation with other reference objects, or in the case of their absence, the input word / word group is said to be “not a term” (kOnt=0).
Thus, the process of the terminology degree evaluation of the input word group is a movement along the graph, at the nodes of which there are objects of the corresponding ontology classes.
In addition to the terminology degree evaluation of a word / word group, the developed metrics makes it possible to extract terms from the text by comparing them with the existing objects and combinations of lemmas of the corresponding objects with the help of Radd ratios, which expand the set of objects of the described subject domain by combining lemmas of related objects. For example: the properties “has a Link” and “is a Part”.
Thus, while comparing the input word groups and the objects of the subject domain, which are related to each other by unidirectional relations Radd, the word group will be said to belong to a certain subject domain, if its lemma completely corresponds to the set of lemmas of the corresponding ontology objects.
The nested link method makes it possible to extract terms from the text by comparing them with the existing objects and combinations of lemmas of the corresponding objects using the links defined in the ontology.
The scheme of the algorithm is shown in Fig. 4. The peculiarity of this method is the necessity to represent the ontology objects mainly in the form of single words with the maximum number of links among objects. The determining factors for this method are Radd, links, and it is possible to form word combinations in a natural way using these links.
t1 + R1 + t2 + R2 + … + ti + Rj + … + tm + Rn, where Ri ∈ Radd, tj ∈ T ,T are the terms of the application area, which the ontology describes.
In this case, the extracted terms, which in its turn belong to terms consisting of a greater number of
words, are not considered as terms to avoid redundancy [
In the course of the research, the linguistic ontology “Cardiovascular diseases” was developed for further recognition of the continuous speech, namely dictated medical diagnoses for this research area.
The ontology has 4 levels of hierarchy, and contains 743 terms belonging to the subject domain. Most of these words were automatically extracted from the texts using the nested link algorithm.
Figure 6 shows a fragment of the developed ontology that contains a description of all the phonemes used in the pronunciation of the extracted terminology.
Moreover, the probabilities of phoneme occurrence, which were found in the extracted terms of the subject domain under consideration, were calculated according to (3). These values will limit the set of selected phonemes while reconstructing the phonetic composition of the recognized words, the speech signal is divided into.
The use of lexical ontology implies the possibility to determine the basic set of terms for the analysed subject domain and, as a result, to increase the probability of accurate determination of certain phonetic combinations sequence in the process of speech recognition.
Within the framework of this research, it is planned to carry out a number of experiments to reconstruct the phonetic composition of recognized words using the developed model of the lexical ontology “Cardiovascular diseases” in order to validate the approach effectiveness.