This paper provides a rough description of zero resource Query-by-Example retrieving system for the MediaEval 2013 spoken web search task. The proposed solution firstly implements the voice activity detection (VAD) utilizing variance of acceleration MFCC (VAMFCC) rule-based approach. A PCA-based segmentation, K-means clustering and GMM training are then used in order to built the posteriorgrams. Finally, two searching architectures based on posteriorgram matching (SDTW) and GMM modeling (GMM-FST) are evaluated. Results show that none of our systems is able to achieve the positive Actual Term Weighted Value, because of high number of insertions. We suppose that chosen clustering scheme caused generation of too many false alarms. Only provided data were used and no other resources were examined in any system component during the development.
The main purpose of our experiments was to check the proposed approaches for the language independent audio query detection and new speech feature analysis components. The mentioned approach is used in the MediaEval activity [1] and could be also applied in various speech [4] or non-speech [5] Query-by-Example applications.
Proposed solution for SWS task uses posteriorgram term matching and audio segment GMM modeling. The overall architecture of proposed system is depicted on Fig. 1.
At first, a training phase is carried out using available development utterances. The VAMFCC-based silence detector performs the initial discrimination of silent parts in audio stream. The block of feature extraction is implemented after VAD utilizing 13 MFCCs. The phase of segmentation and clustering creates the audio segment units (ASU). ASU is represented as a small audio part (phoneme for example) with some spectral and temporal characteristics, different for each ASU. Then the training of acoustic models is performed using these ASU, where each ASU represents one class. Labels for these classes (ASUs) are assigned according to the number of GMM. Only the process of voice activity detection and feature extraction is then performed in preprocessing stage within the retrieving phase. Each utterance and
Copyright is held by the author/owner(s). Segmental Dynamic Time Warping (SDTW) algorithm is then used for comparing these posteriorgrams and finding a possible occurrence of a query in the test utterance. The other solution GMM-FST (GMM-Finite State Transducers) implements a Viterbi algorithm to create a model for each query and state sequence network to find occurrences in test utterance by using this model.
In order to identify and to distinguish the speech segments in the i-th utterance, PCA (Principal Component Analysis) was applied as follows. Each 13-dimensional MFCC vector xj was reshaped to matrix Xj with row dimension nr, where j ∈ 1; ni is the number of vectors in i-th recording. In the next step, the covariance matrix C1 was computed from the first matrix X1 and its eigenvectors and eigenvalues were computed. The eigenvalue spectrum Λ1 = {λ1j} nr j=1 was used to determine the significance ∆(λ1 max ) of the dominant eigenvalue of C1 as ∆(λ1 max ) =
, where λ1j are the eigenvalues of C1. Then the matrix X1 was spliced together with X2 and the covariance matrix C12 and ∆(λ12 max ) were computed again. If ∆(λ12 max ) compared to ∆(λ1 max ) changed significantly, a new speech segment was created and PCA started from the current frame. In the other way, if ∆(λ12 max ) did not change significantly, the current matrix X12 was spliced together with X3 and the process was repeated automatically until a new segment was indicated. The created segments corresponded to ASUs. In the next phase, the segments with similar acoustic and statistical properties were grouped together into several speech clusters using k-means clustering with k = 50 clusters and squared Euclidean distance metrics. As the input data for clustering the means of the segments were used. Each mean vector obtained an index (label) of the specific cluster. This label was assigned to the original feature vectors corresponding to the specific mean vector.
A retrieving process uses Weighted Finite State Transducers (WFST) that allow us to find the most probable path (state sequence) in search network [2]. The search of a query consists of two steps. At first, query alone is recognized using search network, created from the trained acoustic model so that all GMM states are arranged in parallel. The result is a sequence of states that model the particular query. The process of recognition is done repeatedly with different insertion penalties in order to obtain multiple states sequences with different lengths. It helps to improve the model representation of retrieving query. The sequences are labeled and added to the previous search network in parallel. The second step involves the recognition of a test utterance using Viterbi algorithm. The final score for decision is computed as a difference between modeled likelihoods of query and utterance using lambda acoustic model where score = (P (Ooccurence|λ) − P (Oquery|λ)) + Θ. Score is then shifted by predefined value Θ and then results with score below zero are removed.
A simplified SDTW searching algorithm was utilized in our system, similar to that used in [3]. The adjustment window condition was set to |(i k − i1) − (j k − j1)| ≤ R, where i1 and j1 are starting coordinates of warping path in each segment, i k and j k define the k-th coordinates and R represents the constraint parameter, set to M/2, where M is the length of query. The range of starting coordinates was conditioned by the constraint parameter and length of each utterance:
2R+1 . The process of finding the optimal local alignment between each utterance and query produces a set of local warp paths, equal to the number of diagonal regions. A score parameter was then set in the following form score
,where n is the number of steps in local alignment, N is the length of utterance and M the length of query, n 1 warpDist represents a summation of components in each warping path, where components are computed from Bhattacharyya distance matrix.
The official results for SWS task are listed in Tab. 1. Two metrics were used to asses the overall performance of GMM-FST and SDTW on dev and eval queries: the actual AT W V , normalized Cnxe and minimal cross-entropy C min nxe . The score normalization for both systems was performed only on development data. A minimum-cost alignment (MCA) for each segment was used as detection score at first level of search in case of SDTW. Final detection of retrieved query was then carried out utilizing score parameter defined in (2.2.2). A threshold for this parameter was set to 0.0819. A decision threshold for score parameter was set to 2.8 in case of GMM-FST based system, while Θ = 3 (2.2.1). Both systems produced a huge amount of false alarms (FA) during the evaluation. Regarding the evaluation results, the GMM-FST system is more appropriate solution for SWS task, because of its lower tendency to detect spurious terms.
All the experiments were mainly done using 2x IBM System x3650 servers, 2x Intel Xeon QuadCore E5530 CPU @ 2.4 GHz Hyper-threading enabled (16 threads), 28 GB RAM, 1TB SAS HDD (RAID5), running Debian OS.
The Speed Factors (the ratio of the total time employed in searching{indexing} the set of queries in{and} the set of audio documents to the product{sum} of their total durations) and Peak Memory Usage during Searching{Indexing} tasks are presented in Tab. 2. The Performance Load equals 0.9 • SSF • P M US + 0.1 • ISF • P M UI is derived from them.
In the future, an improved clustering and segmentation algorithm will be investigated in order to decrease the overlapping between individual ASUs. A minimal length of warping path algorithm will be integrated in SDTW approach, too.
This research was supported by the ERDF funded projects ITMS-26220220155 (50%) & ITMS-26220220182 (50%).