=Paper=
{{Paper
|id=None
|storemode=property
|title=TUKE at MediaEval 2013 Spoken Web Search Task
|pdfUrl=https://ceur-ws.org/Vol-1043/mediaeval2013_submission_73.pdf
|volume=Vol-1043
|dblpUrl=https://dblp.org/rec/conf/mediaeval/VavrekPLVKHJ13
}}
==TUKE at MediaEval 2013 Spoken Web Search Task==
https://ceur-ws.org/Vol-1043/mediaeval2013_submission_73.pdf
TUKE at MediaEval 2013 Spoken Web Search Task
Jozef Vavrek, Matúš Pleva, Martin Lojka, Peter Viszlay,
Eva Kiktová, Daniel Hládek, Jozef Juhár
Technical University of Kosice, Letna 9, 04200 Košice, Slovakia
{Jozef.Vavrek, Matus.Pleva, Martin.Lojka, Peter.Viszlay,
Eva.Kiktova, Daniel.Hladek, Jozef.Juhar}@tuke.sk
TRAINING PHASE
ABSTRACT Dev. VAD Segmentation K-means GMM Acoustic
MFCC (13)
utterances VAMFCC based training
This paper provides a rough description of zero resource PCA based clustering models
Query-by-Example retrieving system for the MediaEval 2013
spoken web search task. The proposed solution firstly imple-
RETRIEVING PHASE
ments the voice activity detection (VAD) utilizing variance Posteriorgram
Audio query
of acceleration MFCC (VAMFCC) rule-based approach. A VAD
MFCC (39)
Scoring using for query
Posteriorgram
SDTW
VAMFCC based Mah. distance
PCA-based segmentation, K-means clustering and GMM Test utterance for utterances
training are then used in order to built the posteriorgrams. Transducer
Final match
GMM based
Finally, two searching architectures based on posteriorgram
matching (SDTW) and GMM modeling (GMM-FST) are
evaluated. Results show that none of our systems is able to Figure 1: SWS framework architecture
achieve the positive Actual Term Weighted Value, because
of high number of insertions. We suppose that chosen clus-
tering scheme caused generation of too many false alarms.
query enters the block of model-based scorer, where scoring
Only provided data were used and no other resources were
is performed by computing the Mahalanobis distance be-
examined in any system component during the development.
tween MFCC vectors (frames with length 25 ms and 10 ms
shift) and GMM acoustic model. The product of that are
1. MOTIVATION posteriorgrams. They are defined as probability vectors with
The main purpose of our experiments was to check the the length of N, where N represents the number of GMMs.
proposed approaches for the language independent audio Segmental Dynamic Time Warping (SDTW) algorithm is
query detection and new speech feature analysis compo- then used for comparing these posteriorgrams and finding
nents. The mentioned approach is used in the MediaEval a possible occurrence of a query in the test utterance. The
activity [1] and could be also applied in various speech [4] other solution GMM-FST (GMM-Finite State Transducers)
or non-speech [5] Query-by-Example applications. implements a Viterbi algorithm to create a model for each
query and state sequence network to find occurrences in test
utterance by using this model.
2. SYSTEM OVERVIEW
Proposed solution for SWS task uses posteriorgram term 2.1 Segmentation and Clustering
matching and audio segment GMM modeling. The overall In order to identify and to distinguish the speech segments
architecture of proposed system is depicted on Fig.1. in the i-th utterance, PCA (Principal Component Analysis)
At first, a training phase is carried out using available de- was applied as follows. Each 13-dimensional MFCC vec-
velopment utterances. The VAMFCC-based silence detector tor xj was reshaped to matrix Xj with row dimension nr ,
performs the initial discrimination of silent parts in audio where j ∈ h1; ni i is the number of vectors in i-th record-
stream. The block of feature extraction is implemented af- ing. In the next step, the covariance matrix C1 was com-
ter VAD utilizing 13 MFCCs. The phase of segmentation puted from the first matrix X1 and its eigenvectors and
and clustering creates the audio segment units (ASU). ASU eigenvalues were computed. The eigenvalue spectrum Λ1 =
is represented as a small audio part (phoneme for example) {λ1j }n r
j=1 was used to determine the significance ∆(λ1max )
with some spectral and temporal characteristics, different for λ
each ASU. Then the training of acoustic models is performed of the dominant eigenvalue of C1 as ∆(λ1max ) = Pn1rmax
λ
,
j=1 1j
using these ASU, where each ASU represents one class. La- where λ1j are the eigenvalues of C1 . Then the matrix X1
bels for these classes (ASUs) are assigned according to the was spliced together with X2 and the covariance matrix C12
number of GMM. Only the process of voice activity detec- and ∆(λ12max ) were computed again. If ∆(λ12max ) com-
tion and feature extraction is then performed in preprocess- pared to ∆(λ1max ) changed significantly, a new speech seg-
ing stage within the retrieving phase. Each utterance and ment was created and PCA started from the current frame.
In the other way, if ∆(λ12max ) did not change significantly,
the current matrix X12 was spliced together with X3 and
Copyright is held by the author/owner(s). the process was repeated automatically until a new segment
MediaEval 2013 Workshop, October 18-19, 2013, Barcelona, Spain was indicated. The created segments corresponded to ASUs.
Table 1: Evaluation results of the tested algorithms Table 2: Processing resources measures
min
query system ATWV Cnxe Cnxe system ISF SSF P M UI P M US PL
dev GMM-FST -0.1371 0.980745 0.976363 GMM-FST 0.0054 0.0048 2GB 1.8GB 0.009
eval GMM-FST -0.1372 0.98505 0.979883 SDTW 0.0054 0.0046 2GB 2.2GB 0.01
dev SDTW -0.4176 0.998421 0.989649
eval SDTW -0.4252 0.998494 0.988404
3. EXPERIMENTAL RESULTS AND CON-
CLUSIONS
In the next phase, the segments with similar acoustic The official results for SWS task are listed in Tab. 1.
and statistical properties were grouped together into sev- Two metrics were used to asses the overall performance of
eral speech clusters using k-means clustering with k = 50 GMM-FST and SDTW on dev and eval queries: the ac-
clusters and squared Euclidean distance metrics. As the in- tual AT W V , normalized Cnxe and minimal cross-entropy
min
put data for clustering the means of the segments were used. Cnxe . The score normalization for both systems was per-
Each mean vector obtained an index (label) of the specific formed only on development data. A minimum-cost align-
cluster. This label was assigned to the original feature vec- ment (MCA) for each segment was used as detection score at
tors corresponding to the specific mean vector. first level of search in case of SDTW. Final detection of re-
trieved query was then carried out utilizing score parameter
2.2 Searching techniques defined in (2.2.2). A threshold for this parameter was set to
0.0819. A decision threshold for score parameter was set to
2.2.1 GMM approach 2.8 in case of GMM-FST based system, while Θ = 3 (2.2.1).
Both systems produced a huge amount of false alarms (FA)
A retrieving process uses Weighted Finite State Transduc-
during the evaluation. Regarding the evaluation results, the
ers (WFST) that allow us to find the most probable path
GMM-FST system is more appropriate solution for SWS
(state sequence) in search network [2]. The search of a query
task, because of its lower tendency to detect spurious terms.
consists of two steps. At first, query alone is recognized us-
All the experiments were mainly done using 2x IBM Sys-
ing search network, created from the trained acoustic model
tem x3650 servers, 2x Intelr Xeonr QuadCore E5530 CPU
so that all GMM states are arranged in parallel. The re-
@ 2.4 GHz Hyper-threading enabled (16 threads), 28 GB
sult is a sequence of states that model the particular query.
RAM, 1TB SAS HDD (RAID5), running Debian OS.
The process of recognition is done repeatedly with differ-
The Speed Factors (the ratio of the total time employed
ent insertion penalties in order to obtain multiple states
in searching{indexing} the set of queries in{and} the set of
sequences with different lengths. It helps to improve the
audio documents to the product{sum} of their total dura-
model representation of retrieving query. The sequences are
tions) and Peak Memory Usage during Searching{Indexing}
labeled and added to the previous search network in paral-
tasks are presented in Tab. 2. The Performance Load equals
lel. The second step involves the recognition of a test ut-
0.9 · SSF · P M US + 0.1 · ISF · P M UI is derived from them.
terance using Viterbi algorithm. The final score for decision
In the future, an improved clustering and segmentation al-
is computed as a difference between modeled likelihoods of
gorithm will be investigated in order to decrease the overlap-
query and utterance using lambda acoustic model where
ping between individual ASUs. A minimal length of warping
score = (P (Ooccurence |λ) − P (Oquery |λ)) + Θ. Score is then
path algorithm will be integrated in SDTW approach, too.
shifted by predefined value Θ and then results with score
below zero are removed.
4. ACKNOWLEDGMENTS
2.2.2 SDTW detection This research was supported by the ERDF funded projects
A simplified SDTW searching algorithm was utilized in ITMS-26220220155 (50%) & ITMS-26220220182 (50%).
our system, similar to that used in [3]. The adjustment win-
dow condition was set to |(ik − i1 ) − (jk − j1 )| ≤ R, where i1 5. REFERENCES
and j1 are starting coordinates of warping path in each seg- [1] X. Anguera, F. Metze, A. Buzo, I. Szoke, and L. J.
ment, ik and jk define the k-th coordinates and R represents Rodriguez-Fuentes. The Spoken Web Search Task. In
the constraint parameter, set to M/2, where M is the length MediaEval 2013 Workshop, page 4, Barcelona, Spain,
of query. The range of starting coordinates was conditioned 18-19 October 2013.
by the constraint parameter and length of each utterance: [2] M. Lojka and J. Juhár. Finite-state transducers and
N −1 speech recognition in Slovak language. In SPA 2009
((2R + 1)k + 1, 1), where 0 ≤ k ≤ 2R+1
. The process of
Conference, pages 149–153, art. no. 5941305, 2009.
finding the optimal local alignment between each utterance [3] A. Park and J. Glass. Unsupervised pattern discovery
and query produces a set of local warp paths, equal to the in speech. IEEE T Audio Speech, 16(1):186–197, 2008.
number of diagonal regions. A score
s parameter was then set [4] J. Vavrek, M. Pleva, and J. Juhár. TUKE MediaEval
P
2n
n warpDist
1
2012: Spoken Web search using DTW and
in the following form score = N +M n+1
,where unsupervised SVM. In MediaEval 2012 Workshop, Pisa
n is the number of steps in local alignment, - CEUR Workshop Proceedings, volume 927, 2012.
P N is the length
of utterance and M the length of query, n [5] E. Vozáriková, M. Pleva, S. Ondáš, J. Vavrek, J. Juhár,
1 warpDist rep-
resents a summation of components in each warping path, and A. Čižmár. Detection and classification of audio
where components are computed from Bhattacharyya dis- events in noisy environment. Journal of Computer
tance matrix. Science and Control Systems, 3(1):253–258, 2010.