In this paper, we attempt to resolve the Spoken Term Detection problem for under-resourced languages within the Automatic Speech Recognition (ASR) paradigm. The proposed methods are validated with unseen dataset in multiple languages. partially with the Romanian phonemes subspaces. For the Romanian database the ASR system yields a Phone Error Rate (PhER) of 36.8%. At this point, all the queries and the contents are transcribed by using the adapted ASR and they are passed to the search block for Spoken Term Detection (STD). If the ASR accuracy would be 100% then the STD is reduced to a simple character string search of a query within a textual content. As the experimental results show, we are far from the ideal case, hence we have to find within a content a string which is similar to the query. The search of the exact query string has poor STD results: 99% Miss Proability (MP) and 0.1% False Alarm Probability (FAP). Moreover, it does not offer the possibility to find a compromise between MP and FAP. The DTW String Search (DTWSS) uses the Dynamic Time Warping to align a string (a query) within a content. The search is not performed on the entire content, but only on a part of it by the means of a sliding window proportional to the length of the query. The term is considered detected if the DTW scores above a threshold. This method is refined by introducing a penalization for the short queries and the spread of the DTW match. The formula for the score s is given by equation (1):
We approach the Spoken Web Search (SWS) Task @ MediaEval
2013 [
For this task, we use the ASR system for the Romanian language
that we have previously developed and described in [
The trained Hidden Markov Models divide speech feature space into phoneme classes and phonemes that do not belong to the Romanian language will be classified into one of these classes. If all instances of an out-of-Romanian-language phoneme are recognized as a given Romanian phoneme, than the problem is solved, because both queries and contents will have the same symbol. However, this is not always the case and the experimental results confirm this. It seems that many phonemes occupy a region in the speech feature space that overlap only s (1 PhER)(1
LQM LQm LQ LQm )(1 LW LS )
LQ (1) where LQ is the length of the query, LQM=17 and LQm=4 are the maximum and the minimum query lengths found in the development data set, LW is the length of the sliding window, LS is the length of the matched term in the content, while α and β are the tuning parameters.
The penalizations in formula (1) are motivated by the assumption that for two queries of different length that match their respective contents by the same PhER, the match of the longer query is more probable to be the right one. Similarly the more compact DTW matches are assumed to be more probable than the longer ones.
We started from the Romanian ASR which had PhER of 36.8% (tested on Romanian data). After tuning the beam width related parameters we succeeded in reducing PhER to 31.4%. The tuning of language related parameters (language weight and word insertion penalty) brought a further reduction of PhER to 25.3%.
The results obtained in the official run for the primary method on the evaluation data set is shown in Figure 1. The primary metric used for comparison is Maximum Term Weighted Value (MTWV). For other values of α and β similar curves are obtained. Depending on the application a trade-off can be made between false alarms and miss rates.
The effect of weighting the score according to the query length and the spread of the alignment match is given by the results presented in Table 1 which are obtained with the development data. These results are obtained for a sliding window length equal to 1.5 times the query length. The shorter the query, the greater are the chances that different, but similar words obtain higher scores. This is why better results are obtained by giving α a greater value. Similarly, the greater the spread of the DTW match, the lower the probability that it is the searched term. However, there is an optimal value for both α and β. The optimal values for α and β are explained by the fact that by increasing α above a certain value the shorter queries are given higher scores just for being short, even though they might have a great PhER value. The same rationale takes place for the spread of the DTW match for which the β factor is responsible. The parameters for the 3 DTWSS official runs are chosen based on these values. ATWV The official runs results for all combinations of testing data sets (development data, evaluation data) are shown in Table 2. All the methods suffer performance degradation when moving from training data to unseen data. However, the degradation is not drastic. This is explained by the fact that development data are used only for tuning the α and β parameters but not for adapting the ASR system. Overall, the results are poor which means that the trained Romanian phonemes do not divide optimally the speech feature space. Phonemes from other languages lie in regions between two or more classes and different instances of the same phoneme are classified differently, thus increasing the uncertainty in the decision process. The system can be significantly improved phonemes from other languages can be trained and introduced in the phone recognizer.
Given the simplicity of the searching component the real time factor is very low (6*10-5 s-1). Memory used during the process is 7.1 GB.
We have approached STD with a two step process. A Romanian ASR is used as a phone recognizer for indexing the database, while a DTW based algorithm is used for searching a given query in the content database. The results are improved if the decision score is weighted according to the length of the query and the spread of the alignment match.