The INESC-ID's Spoken Language Systems Laboratory (L2F) primary system developed for the Spoken Web Search task of the Mediaeval 2013 evaluation campaign consists of the fusion of six individual sub-systems exploiting 3 di erent language-dependent phonetic classi ers. For each phonetic classi er, an acoustic keyword spotting (AKWS) sub-system based on connectionist speech recognition and a dynamic time warping (DTW) based sub-system have been developed. The diversity in terms of phonetic classi ers and methods, together with the e cient fusion and calibration approach applied for heterogeneous sub-systems, are the key elements of the L2F submission. Besides the primary submission, two additional systems based on the fusion of only the AKWS and the DTW sub-systems have been developed for comparison purposes. A nal multi-site system formed by the fusion of the L2F and the GTTS primary submissions has been also submitted to explore the potential of the fusion approach for very heterogeneous systems.
This document introduces the Spoken Web Search systems
developed by the INESC-ID's Spoken Language Systems
Laboratory (L2F) for the Mediaeval 2013 campaign. The
targeted task in this challenge is query-by-example spoken
term detection. Detailed information about the task and
the data used can be found in the evaluation plan [
Six sub-systems form the core of the L2F SWS system exploiting three di erent language-dependent phonetic networks trained for European Portuguese (pt ), Brazilian Portuguese (br ) and European Spanish (es). The phonetic networks are used either as acoustic models in acoustic KWS based on hybrid connectionist methods or as a feature extraction component for DTW based term detection.
L2F systems exploit multi-layer perceptron (MLP) networks
that are part of our in-house hybrid connectionist ASR
system. The phonetic class posterior probabilities are in fact
the result of the combination of four MLP outputs trained
with Perceptual Linear Prediction features (PLP, 13 static
+ rst derivative), PLP with log-RelAtive SpecTrAl speech
processing features (PLP-RASTA, 13 static + rst
derivative), Modulation SpectroGram features (MSG, 28 static)
and Advanced Font-End from ETSI features (ETSI, 13 static
+ rst and second derivatives). The language-dependent
MLP networks were trained using di erent amounts of
annotated data [
AKWS sub-systems exploit the phonetic networks as
acoustic models for both phonetic tokenization and query search
based on hybrid ANN/HMM approaches for ASR [
DTW sub-systems use the language-dependent phonetic networks to extract log-posterior features. The silence class of the phonetic network is also used for voice activity detection. To this end, the segments identi ed as silence at the beginning and end of each query and document are removed. For each query-document pair, N euclidean distance based DTWs are run on N starting candidate positions of the document. To select the candidate positions, the querydocument euclidean distance matrix of the DTW is used. The minimum of each column of the matrix represents the minimum distance among all query feature vectors to a given document feature vector. The average of these minima on a sliding window of query size is used as an approximation of DTW without the warping constraints, from which the best N candidates are selected. The number of candidates N was made equal to the length of the document in feature vectors divided by 100 with a minimum of 100 candidates. In a second stage, DTWs of the size of the query are evaluated at each one of the N candidate positions, and the three candidates with lower normalized cumulative distance, and separated by at least 0.5 seconds, are kept. The reduction of the search space to N candidates as explained above provided a reduction of the search time by a factor of around 5, while having a minimal impact on the performance. It should be noted that the DTW, including the distance matrix, was computed using the R programming language, while the candidate selection and remaining tasks were implemented in Python1. This framework bene ted particularly from the candidate selection scheme proposed.
The combination of systems is based on a recently
proposed method for discriminative calibration/fusion of
heterogeneous spoken term detection (STD) systems [
One primary and two contrastive \on-time" systems were
submitted. The primary system consists of the fusion of the
six sub-systems previously described, while the contrastive1
and contrastive2 submissions correspond to the fusion of
only the DTW and only the AKWS sub-systems,
respectively. Additionally, a \late" contrastive3 system based on
the fusion of the primary systems of the L2F and GTTS[
This work was partially funded by the DIRHA European project (FP7-ICT-2011-7-288121) and the Portuguese Foundation for Science and Technology (FCT), through the projects PEst-OE/EEI/LA0021/2013 and PTDC/EIA-CCO/122542/ 2010, and the grant number SFRH/BPD/68428/2010.