The problem of increasing the efficiency of parametric identification of Gaussian mixture model (GMM) is considered. A method for identifying GMM parameters based on clonal selection algorithm with preliminary formation of a learning set taking into account the structure of vocal sounds, which increases the likelihood of speaker recognition, is proposed. The parameter identification method is intended for software implementation on the GPU using the CUDA technology, which speeds up the process of parametric identification. This method has been studied on the TIMIT database and serves for intelligent systems of biometric personal identification.
Automated biometric identification of a person means making decisions based on
acoustic and visual information, which improves the quality of recognition of the
person under study. Unlike the traditional approach [
A special class of biometric identification of a person is formed by methods based
on the analysis of acoustic information [
The well-known voice identification methods, such as dynamic programming
[34], analyze the entire signal, which increases the likelihood of recognition, but require
entire storage of learning signals and long comparison of the analyzed signal with all
learning signals. Other well-known methods, such as vector quantization [
The aim of the work is to increase the efficiency of parametric identification method of Gaussian mixture model (GMM) due to a metaheuristic clonal selection algorithm, with preliminary formation of a learning set.
To achieve this goal, it is necessary to solve the following tasks:
2
3
Biometric identification methods based on dynamic programming [
Biometric identification methods based on vector quantization [
Biometric identification methods based on artificial neural networks [
Biometric identification methods based on decision trees [
Biometric identification methods based on Gaussian mixture models (GMMs) [1013] identify GMM parameters, which store information about signal components representing the well-known speakers, and recognize the signal representing the recognizable speaker by means of these GMMs. The advantage of these methods lies in the ability to quickly recognize the entire signal and store only GMM parameters. The disadvantage of these methods consists in the identification of GMM parameters based on local search, which reduces the likelihood of recognition.
A common feature of all methods listed above consists in the following: they form a learning set of signal patterns and analyze a signal without taking into account the structure of vocal sounds, which leads to a decrease in the probability of recognition.
Therefore, the increase of the efficiency of the method of GMM parametric identification with preliminary formation of a learning set, taking into account the structure of vocal sounds, is an urgent task. 4
The method of learning set formation includes the following steps: ─ partitioning of a quasi-periodic signal into discrete patterns; ─ shifting of discrete patterns in time and amplitude; ─ interpolation of discrete patterns; ─ shifting and scaling of continuous patterns in time; ─ shifting and scaling of continuous patterns in amplitude; ─ sampling of continuous patterns.
Let’s define a finite set of discrete patterns of a quasi-periodic signal, described by a family of integer bounded finite discrete functions X {xi | i {1,..., I}} , in the form:
f (n), n {Nimin ,..., Nimax } xi (n) 0, n {Nimin ,..., Nimax }
, i {1,..., I} , Amin min f (n) , n {Nimin ,..., Nimax } , i {1,..., I} ,
i n Amax max f (n) , n {Nimin ,..., Nimax } , i {1,..., I} ,
i n where Amin , Aimax – the minimum and maximum values of the function xi on the i compact {Nimin ,..., Nimax } .
Let’s define a finite set of discrete patterns shifted in time and amplitude, which are described by a finite family of integer bounded finite discrete functions X s {xis | i {1,..., I}} , in the form: xis (n) xi (n Nimin ) Aimin , n {0,..., Ni } , i {1,..., I} , 0, n {0,..., Ni }
Ni Nimax Nimin , Ai Aimax Aimin .
A linear interpolation, which requires the least computational complexity, has been chosen in the article. Let’s define a finite set of continuous patterns, obtained as a result of linear interpolation and described by a finite family of real-valued bounded finite continuous functions { i | i {1,..., I}} , in the form:
t [t,Ti t] i (t) (tn ,tn1) (t) xis (n) xis (n 1) xis (n) (t tn ) {tn}(t)xis (n) , Ni Ni 1 n1 t n1 i {1,..., I} , t [t,Ti t] i (t) 0 , Ti Ni t , tn nt ,
1, t B B (t) 0, t B
– indicator function, where t – the quantization step in time.
Let’s define a finite set of shifted and scaled in time continuous patterns, described by a finite family of real-valued bounded finite continuous functions s { is | i {1,..., I}} , in the form: t Tmin t [Tmin ,Tmax ] is (t) i Ti Tmax Tmin ,
t [Tmin , Tmax ] is (t) 0 , where [Tmin ,Tmax ] – the compact, set and single for all patterns.
The article proposes to define Tmin ,Tmax as follows:
Tmin t , Tmax round fd t ,
fmin where fd – the sampling frequency in Hz (for vocal sounds 8 kHz is enough), fmin – the minimum frequency of frequency range of speech sound (for vocal sounds 50 Hz is enough), round () – the function, rounding the number to the nearest integer.
Let’s define a finite set of continuous patterns shifted and scaled in amplitude, which are described by a finite family of real-valued bounded finite continuous functions ss { iss | i {1,..., I}} , in the form: t [Tmin ,Tmax ] iss (t) Amin
Amax Amin A max Aimin is t ,
i t [Tmin ,Tmax ] iss (t) 0 ,
A max max is t , i t A min min is t , i t t [Tmin ,Tmax ] , where A min , Aimax – the minimum and maximum values of the function is on the i compact [Tmin ,Tmax ] ; Amin , Amax – the minimum and maximum values, set and single for all patterns, on a given compact [Tmin ,Tmax ] .
The article proposes to define Amin , Amax as follows: Amin 0 , Amax 2b 1 , where b – the number of level quantization bits.
Let’s define a finite set of discrete patterns, obtained from continuous ones by sampling in time and described by a finite family of integral bounded finite discrete functions S {si | i {1,..., I}} , in the form:
si (n) round ( iss (nt)) , n {N min ,..., N max } ,
N min Tmin / t , N max Tmax / t , N N max N min .
Each received discrete sample is considered as a feature vector and is located in a single amplitude-time window. 5
Each GMM associated with a particular speaker is defined on the basis of the total probability formula as follows:
K P(s) P(k ) p(s | k ) ,
k 1 p(s | k )
1 p(s | k ) – densities of multi-dimensional Gaussian distribution, mk (mk1,..., mkN ) – vector of mathematical expectations, Ck – covariance matrix, K – the number of mixture components.
In the work, for GMM parametric identification the target function, which means the choice of such values of parameter vector that deliver likelihood functions to the maximum logarithm, is chosen:
I K F ln P(S | ) ln P(k) p(si | k) max ,
i1 k 1 where ((P(1),..., P(K )), (m1,..., mK ), (C1,..., CK )) – GMM parameters vector.
Since the traditional EM algorithm used for GMM parametric identification implements only local search, so currently for GMM parametric identification are actively applied evolutionary, swarm and immune metaheuristics, which use the parameter vector as an individual of the population [14-16 ]. However, due to the large dimensionality of the covariance matrix C , only a diagonal matrix C is used in these metaheuristics, which reduces the likelihood of identification. Therefore, in this work, the joint probability vector (P(s1,1),..., P(si , k),..., P(sI , K )) , where P(si , k ) P(k ) p(si | k ) , is used as an individual of the population, which allows to work with the off-diagonal matrix C .
GMM parametric identification is based on clonal selection algorithm proposed in [17-23] and is presented in the following form: 1. The number of mixture components K , the mutation parameter , the maximum number of iterations max are set. 2. The intermediate population U u of the power is created, each antibody of the population being represented as u (u11,..., uIK ) .
Each element uik is defined as uik rand () , where rand () – a function that
returns a uniformly distributed random number in the range [
A finite set of discrete patterns S is given. 3. The iteration number 1 is set, the maximum value of the target function 4. A posteriori conditional probabilities for each antibody u are calculated: 7. Covariance matrices for each antibody u are calculated:
I P(k | si )si mk i1I P(k | si ) i1
, k {1,., K} .
Ck i1 I P(k | si ) si m T si m
k k I P(k | si ) i1 , k {1,., K} . 8. The densities of multidimensional Gaussian distribution for each antibody u are calculated: p(si | k )
1 (2 )N det Ck
1 exp (si m )T Ck1(si mk ) , k {1,., K} , i {1,..., I} .
2 k
K 9. The total probabilities for each antibody u are calculated P(si ) P(k) p(si | k) . k 1 10. The value of the target function for each antibody u is calculated:
I F (u) ln P(si ) .
i1 11. The intermediate population U u is ordered in descending of the target function value. 12. The current population H h of the power is created by means of reproduction and replacement operators, with each antibody of this population being represented as h (h11,..., hIK ) .
As antibodies of the current population H h , the first (the best in the target function) antibodies of the intermediate population U u are taken. This corresponds to the reduction operator with a selection scheme that provides the search directionality (the best antibodies are preserved). Replacement (randomly generated) antibodies can be among the best antibodies. This corresponds to the replacement operator. 13. The value of the target function of the first antibody of the current population
H h is set by the minimum value of the target function a . 14. The value of the target function of the last antibody of the current population
H h is set by the maximum value of the target function b . 15. If | b bold | and max , then 1 , bold b , going to 16, otherwise parametric identification is complete. 16. The affinity for each antibody h is calculated.
The affinity determines the proximity of the current antibody to the best one and is calculated based on the utility function in the form: 17. The mutation probability for each antibody h is calculated p(h) exp (h) .
The larger , the less likely is the mutation probability. 18. A set of clones C c of the power is created by means of a cloning operator, each clone of this set being represented as c (c11,..., cIK ) .
The cloning operator plays a role similar to the reproduction operator of genetic algorithm and is applied to the current population H h . The number of clones for each antibody h is defined as .
19. A set of mutated clones C c of the power is formed by means of the proposed mutation operator, each mutated clone of this set being represented in the form c (c11,..., cIK ) .
The mutation operator allows to obtain new antibodies with sharply different properties from antibody clones.
Each element cik is defined as:
cik , rik p(h) rik rand () , cik round (), rik p(h)
The features of the proposed variant of the mutation operator are the following: it does not require the use of binary potential solutions, i.e. it doesn’t need to convert probabilistic potential solutions into binary ones before the mutation and to convert binary potential solutions into probabilistic ones after the mutation, which reduces the computational complexity of the mutation operator and allows to quickly find a solution. 20. A set of replacement antibodies D d of the power is created, each antibody of the set is represented as d (d11,..., dIK ) .
Each element dik is defined as dik rand () . 21. The intermediate population U u of the power is formed.
The antibodies of the current population H h and a set of mutated clones C c are taken as the first antibodies of the intermediate population U u . The antibodies of a set of the replacement antibodies D d are taken as the last antibodies of the intermediate population U u . Going to 4.
As a result of the method, GMM parameters will be defined. 6
The algorithm for GMM parametric identification based on clonal selection algorithm, which is intended for implementation on the GPU using the CUDA technology, is presented in Fig. 1. This block diagram functions as follows.
Step 1 – Set the number of mixture components K , the mutation parameter , the maximum number of iterations max .
Step 2 – Set the intermediate population U and a finite set of discrete patterns S .
Step 3 – Set the iteration number 1 , the maximum value of the target function bold 0 .
Step 4 – Calculate the total probabilities P(si ) for each antibody u using I K ( ) threads that are grouped into I ( ) one-dimensional blocks. In each block, based on the reduction, a sum from K elements of the form uik is calculated.
Step 5 – Calculate a posteriori conditional probabilities for each antibody u using I K ( ) threads that are grouped into I K N s onedimensional blocks, where N s is the number of threads in the block. Each thread computes P(k | si ) uik P si .
Step 6 – Calculate the sum of a posteriori conditional probabilities qk for each antibody u using I K threads that are grouped into K twodimensional blocks. In each block, based on the reduction, a sum from I elements of the form P(k | si ) is calculated.
Step 7 – Calculate a priori unconditional probabilities (weights of the mixture components) P(k ) for each antibody u using K threads that are grouped into K N s one-dimensional blocks. Each thread computes P(k ) qk I .
Step 8 – Calculate expectation vectors mk for each antibody u using I K ( ) threads that are grouped into K ( ) two-dimensional blocks. In each block, based on the reduction, a sum from I elements of the form P(k | si )si qk is calculated.
Step 9 – Calculate the covariance matrices Ck for each antibody u using I K ( ) threads that are grouped into K ( ) two-dimensional blocks. In each block, based on the reduction, a sum from I elements of the form P(k | si ) si mk T si mk qk is calculated.
Step 10 – Calculate the densities of Gaussian multidimensional distribution p(si | k) for each antibody u using I K ( ) threads that are grouped into I K N s one-dimensional blocks. Each thread computes: p(si | k)
1
Step 11 – Calculate the total probabilities P(si ) for each antibody u using I K ( ) threads that are grouped into I ( ) one-dimensional blocks. In each block, based on the reduction, a sum from K elements of the form P(k ) p(si | k) is calculated.
Step 12 – Calculate the target function values F (u) for each antibody u using I ( ) threads that are grouped into two-dimensional blocks. In each block, based on the reduction, a sum from I elements of the form ln P(si ) is calculated.
Step 13 – Order the target functions by the value based on parallel sorting by merging into the intermediate population U using threads that are grouped into one one-dimensional block.
1 2 3 23 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
Step 14 – Create the current population H h of the power by means of element-by-element copying of the first antibodies of the intermediate population U u using I K threads that are grouped into I K N s one-dimensional blocks.
Step 15 – Set the value of the target function of the first antibody of the current population H h by the minimum value of the target function a .
Step 16 – Set the value of the target function of the last antibody of the current population H h by the maximum value of the target function b .
Step 17 – If | b bold | and max , then 1 , bold b , going to 18,
Step 19 – Calculate the mutation probability for each antibody h using threads that are grouped into one one-dimensional block p(h) exp (h) .
Step 20 – Create a set of clones C c of the power by means of element-byelement copying of the antibodies of the current population H h using I K threads that are grouped into I K N s one-dimensional blocks.
Step 21 – Form a set of mutated clones C {c} of the power by means of the proposed mutation operator, using I K threads that are grouped into I K N s one-dimensional blocks. Each thread computes:
cik , rik p(h) rik rand () , cik round () rik p(h) .
Step 22 – Create a set of random antibodies D d of the power using I K threads that are grouped into I K N s one-dimensional blocks. Each thread computes dik rand () .
Step 23 – Generate the intermediate population U u of the power by means of element-by-element copying of the antibodies of the current population H h , a set of mutated clones C c , a set of replacement antibodies D d , using I K ( ) threads that are grouped into I K N s onedimensional blocks. Going to 4. 7
Numerical experiments were carried out using the CUDA technology of parallel processing of information on the GeForce 920M video card with the number of threads in the block N s = 1024. Parametric identification was carried out for each of the 100 GMMs, corresponding to 100 speakers. In the work it has been accepted that the power of a set of patterns of vocal speech sounds I 1024 , the power of the current population 20 , the power of a set of clones 1000 , the power of a set of random antibodies 0.2, 4 , the mutation parameter 2.3 , the maximum number of iterations max 1000 , the parameter 106 .
The dependence of the probability of incorrect recognition of the speaker on the number of components of the mixture is shown in Fig.2. This dependence shows that the probability of incorrect recognition of the speaker decreases with increasing number of components of the mixture.
Table 1 presents the speaker’s recognition probabilities obtained from the TIMIT database using a neural network based on radial basis functions (RBFNN), trained on the basis of error correction with the MFCC feature system, Gaussian mixture model (GMM), trained on the basis of EM-algorithm with the MFCC feature system, Gaussian mixture model, trained on the basis of clonal selection algorithm, with features obtained by means of the proposed method of learning set formation.
According to Table 1, the best results are given by GMM, trained on the basis of clonal selection algorithm with features obtained by means of the proposed method of learning set formation.
This is due to the fact that error correction and the EM algorithm only perform a local search, which increases the likelihood of falling into a local extremum, and learning set formation based on the MFCC is done without taking into account the structure of vocal sound. 8
The article deals with the problem of increasing the efficiency of parametric identification of Gaussian mixture model (GMM). The method of learning set formation, which uses shifting, scaling, interpolation and sampling of signal patterns that correspond to quasi-periods to locate them in a single amplitude-time window, which allows to take into account the quasi-periodic signal structure and increase the probability of speaker recognition, has been improved. The method of GMM parametric identification, which is based on clonal selection algorithm that allows to reduce the likelihood of falling into a local extremum, has reached further development. The proposed method allows probabilistic individuals of the population (potential solutions) in the mutation operator, which accelerates parametric identification (it is not necessary to perform the operations of converting real individuals into binary ones and vice versa), and uses not the parameter vector, but the vector of joint probabilities as an individual of the population, which allows to work with a non-diagonal covariance matrix and increases the likelihood of speaker recognition. The algorithm of GMM parametric identification, which is intended for software implementation on the GPU using the CUDA technology that speeds up the GMM learning process, has been created. The software that implements the proposed algorithm has been developed and investigated on the TIMIT database. The experiments have confirmed the operability of the developed software and allow to recommend it for use in practice. Prospects for further research are to test the proposed methods on a wider set of test databases. Procedia Engineering, Cochin, India (2017) doi: doi.org/10.1016/j.procs.2017.09.075 12. Chauhan, V., Dwivedi, Sh., Karale, P., Potdar, S. M.: Speech to text converter using Gaussian Mixture Model (GMM), International Research Journal of Engineering and Technology (IRJET) 3(5), 160–164 (2016) 13. Reynolds, D. A.: Automatic Speaker Recognition Using Gaussian Mixture Speaker Models. The Lincoln Laboratory Journal 8(2), 173–195 (1995) 14. Lin, L., Wang, Sh.: A New Genetically Optimized GMM for Speaker .
In: Published in 6th World Congress on Intelligent Control and Automation, pp. 10235–10239 (2006) doi: 10.1109/WCICA.2006.1714005 15. Zablotskiy, S., Pitakrat, T., Zablotskaya, K., Minker, W.:GMM Parameter Estimation by Means of EM and Genetic Algorithms. In: Human-Computer Interaction. Design and Development Approaches: 14th international conference on Humancomputer interaction: design and development approaches, pp. 527–536. Proceeding, Orlando, FL (2011) doi: 10.1007/978-3-642-21602-2_57 16. Saeidi, R., Mohammadi, H. R. S., Ganchev, T., Rodman, R. D.: Particle Swarm Optimization for Sorted Adapted Gaussian Mixture Models. IEEE Transactions on Audio, Speech, and Language Processing 17(2), 344–353 (2009) doi: 10.1109/TASL.2008.2010278 17. de Castro, L. N., von Zuben, F. J.: The Clonal Selection Algorithm with Engineering Applications. In: Proceedings of the Genetic and Evolutionary Computation Conference (GECCO '00), Workshop on Artificial Immune Systems and Their Applications, pp. 36-39. Las Vegas, Nv (2000) 18. de Castro, L. N., von Zuben, F. J.: Learning and Optimization Using Clonal Selection Principle. IEEE Transactions on Evolutionary Computation 6(3), 239–251 (2002) doi: 10.1109/TEVC.2002.1011539 19. Babayigit, B. A., Guney, K., Akdagli, A.: Clonal Selection Algorithm for Array Pattern Nulling by Controlling the Positions of Selected Elements. Progress in Electromagnetics Research B 6, 257–266 (2008) doi: 10.2528/PIERB08031218 20. White, J. A., Garrett, S. M.: Improved Pattern Recognition with Artificial Clonal Selection? In: Timmis J., Bentley P. J., Hart E. (eds). 2nd International Conference on Artificial Immune Sysyems, vol. 2787, pp. 181–193. Springer, Berlin (2003) doi: 10.1007/978-3-540-45192-1_18 21. Alba, E., Nakib, A., Siarry, P.: Metaheuristics for Dynamic Optimization, Springer-Verlag, Berlin (2013) 22. Du, K.-L., Swamy, M. N. S.: Search and Optimization by Metaheuristics. Techniques and Algorithms Inspired by Nature, Springer, Birkhäuser Basel (2016) doi: 10.1007/978-3-319-41192-7 23. Brownlee, J.: Clever Algorithms: Nature-Inspired Programming Recipes, Melbourne, Brownlee (2012)