An ergodic hidden Markov model (EHMM) of speech can be trained in an unsupervised manner using unlabeled speech. A keyword spotting system has been developed where the queries and test observations are represented as sequences of states of the EHMM. A graphical keyword model is built by aggregating multiple instances of a query or by using mappings between phonemes and states of the EHMM. A modi ed Viterbi algorithm with a 3D lattice structure has been used to score the observations.
Traditional statistical approaches model speech at word or subword level using left-right HMMs. These schemes require knowledge of the linguistic structure of a language and the availability of large amount of labeled data for training. This methodology of speech recognition, therefore, cannot be adopted for a large number of resource-limited languages.
An alternate statistical approach that eliminates reliance on time-aligned labeled training data, is to built a single model of speech. While phonemes always occur in a particular sequence in an utterance, the set of phonemes, in different combinations, form all the utterances in a language. Modeling the entire signal space of a language would require a model with a far more exible structure than that of a left-right HMM.
In this work, a single, large EHMM has been used to model
the entire speech. An EHMM can model non-linear
observation distributions and capture the short-term correlations
in speech. Furthermore, an EHMM can be trained without
labeled data. These characteristics have been the
motivation for the design of an EHMM-based keyword spotting
system for the Spoken Web Search task of MediaEval 2013,
described in detail in [
The design of an EHMM encompasses a number of factors including the size of the EHMM, the form of the observation distributions, initial values of the parameters, and the training methodology.
The training involves the established Baum-Welch
algorithm and can be carried out using the Hidden Markov
Model Toolkit (HTK) [
An EHMM stores su cient detail about the original
signal that perfectly intelligible speech can be synthesized from
its state sequence [
In absence of any time-aligned labeled data for training, observations are grouped into a state based on their proximity in the feature space and exhibition of similar temporal characteristics. Since most feature sets, including MFCCs, are not invariant to all speaker-dependent variations, speech segments with the same perceptual character may occupy distinct regions in the feature space, and consequently, mapped to di erent states of the EHMM.
A graphical keyword model [
A series of experiments were carried out to identify the con guration of system with the highest precision. Due to time constraints, an exhaustive evaluation of every combination of the parameters was not possible. 3.1
The optimal number of states in an EHMM is a function of acoustic variety in the training data and may di er from one language to another. A small number of states may not fully capture the characteristics of the modeled data. Conversely, if the number of states in the EHMM are larger than that required then only a subset of the states will be used for modeling. For the SWS2013 data, a number of di erent EHMMs were tested but larger EHMMS, with number of states greater than 192, lost some of the states during training. Hence, the maximum size of the EHMM was limited to 256 states. The maximum term weighted value (MTWV) was observed for the EHMM with 256 states (Table 1). 3.2
A graphical keyword model can also be built from a single
utterance by incorporating knowledge of similar states of the
EHMM obtained through supervised or unsupervised
clustering schemes. In this work, similar states in the 256-state
EHMM were merged to form superstates or state clusters
using the approach outlined in [
A merger of similar states initially led to an increase in the value of MTWV (Table 2). The EHMM with 192 superstates yielded a higher MTWV than any of the EHMMs trained directly by the Baum-Welch algorithm. 3.3
In these experiments, graphical keyword models were built from multiple instances of a query. Surprisingly, the highest gain in MTWV was observed after the addition of the second example and the precision decreased consistently (Table 3) after the inclusion of each additional example. 3.4
The range of values for log-likelihoods is much larger than
most similarity/distance measures. The histograms of
loglikelihood scores of candidates for two di erent queries,
after normalizing for length di erences, is shown in Figure
2. The large variations in the candidate scores for di erent
trials make the computed TWV, which uses a single scale
for all trials, susceptible to score normalization parameters.
The scores in Tables 1 and 2 were normalized using Z-Norm
while the scores in Table 3 and the nal submissions were
normalized using a variant of T-Norm, as outlined in [
The experiments were carried out on Georgia Tech's heterogeneous PACE cluster, which consists of multi-core (2464 cores, 64-256 GB RAM) Altus-branded servers running RHL6.3. A typical run on the development set would take 40 hours of CPU time with an average memory footprint of 5MB leading to an estimated real-time factor of 0.003. 4.
A zero-resource approach to spoken web search using a single large ergodic hidden Markov model of speech has been presented. The novel graphical keyword model represents a query as a network of states of the EHMM, and can incorporate multiple examples of a keyword or phoneme-to-state mappings. This approach can bene t from improved discovery of phoneme-to-state mappings and the scored TWV can further be increased by a better score normalization scheme.