This paper briefly describes the systems presented by the Software Technologies Working Group (http://gtts.ehu.es, GTTS) of the University of the Basque Country (UPV/EHU) to the Spoken Web Search (SWS) task at MediaEval 2013. GTTS systems consist of four main modules: (1) feature extraction; (2) speech activity detection; (3) DTW-based query matching; and ( 4) score calibration and fusion. The most remarkable contributions are the use of phone loglikelihood ratio features, the normalization of the DTW distance matrix and the calibration/fusion approach (which is imported from language/speaker verification).
The MediaEval 2013 Spoken Web Search (SWS) task consists of searching for a spoken query within a set of audio documents [4]. The locations and durations of all the occurrences of spoken queries in the audio documents must be obtained. System performance is primarily measured in terms of the Average Term-Weighted Value (ATWV) [5], but also in terms of a normalized cross-entropy metric and the processing resources (real-time factor and peak memory usage) required by the submitted systems [6]. For more details on the SWS task at MediaEval 2013, see [2].
The Brno University of Technology (BUT) phone decoders for Czech, Hungarian and Russian [7] are applied to decode both the spoken queries and the audio documents. BUT decoders are trained on 8 kHz SpeechDat(E) databases recorded over fixed telephone networks, containing 12, 10 and 18 hours of speech and featuring 45, 61 and 52 units for Czech, Hungarian and Russian, respectively (three of them being non-phonetic units that stand for short pauses and noises).
Given an input signal of length T , the decoder outputs the posterior probability of each state s
where M is the number of units and S the number of states per unit. The posterior probability of each unit i at each frame t are computed by adding the posteriors of its states:
Finally, the posteriors of the three non-phonetic units are added and stored as a single non-speech posterior. Thus, the size of the frame-level feature vectors is 43, 59 and 50 for the Czech, Hungarian and Russian BUT decoders, respectively.
Given an audio signal, Speech Activity Detection (SAD) is performed by discarding those phone posterior feature vectors for which the non-speech posterior is the highest. The remaining vectors, along with their corresponding time offsets, are stored for further use, but the component corresponding to the non-speech unit is deleted. If the number of speech vectors is too low (in this evaluation, that threshold was arbitrarily set to 10, that is, 0.1 seconds), the whole signal is discarded, to save time and to avoid false alarms.
Given two SAD-filtered sequences of feature vectors corresponding to a spoken query q and a spoken document x, the cosine distance is computed between each pair of vectors, q[i] and x[j] as follows:
Note that d(v, w) ≥ 0, with d(v, w) = 0 if and only if v and w are perfectly aligned and d(v, w) = +∞ if and only if v and w are orthogonal. The distance matrix computed according to Eq. 2 is further normalized with regard to the spoken document x, as follows:
with dmin(j) = min
In this way, matrix values are all comprised between 0 and 1, so that a perfect match would produce a quasi-diagonal sequence of zeroes.
The best match of a query q of length m in a spoken document x of length n is defined as that minimizing the average distance in a crossing path of the matrix dnorm. A crossing path starts at any given frame of x, k1 ∈ [1, n], then traverses a region of x which is optimally aligned to q (involving L vector alignments), and ends at frame k2 ∈ [k1, n]. The average distance in this crossing path is:
where i l and j l are the indices of the vectors of q and x in the alignment l, for l = 1, 2, . . . , L. Note that i1 = 1, iL = m, j1 = k1 and jL = k2. The minimization operation This procedure is iteratively applied to find not only the best match but also less likely matches in the same document. To that end, a queue of search intervals is defined and initialized with (1, n). Let us consider an interval (a, b), and assume that the best match is found at (a , b ), then the intervals (a, a ) and (b , b) are added to the queue (for further processing) if: (1) the score of the current match is greater than a given threshold (in this evaluation, 0.85); (2) the interval is long enough (in this evaluation, half the query length); and (3) the number of matches (already computed + pendant) is less than a given maximum (in this evaluation, 7). Finally, the list of matches for each query is truncated to the N with the highest scores (in this evaluation, N = 1000).
Under the extended (multiple examples) condition, only the examples passing SAD filtering (i.e. with enough speech samples) are considered for each query. The longest example is taken as reference and DTW-aligned to the other available examples. Finally, the vectors aligned at each frame are averaged and a single average example is obtained and processed as in the required (single example) condition.
System scores are transformed according to [1], which is an adaptation of the discriminative calibration/fusion approach commonly applied in speaker and language recognition.
First, the so-called q-norm (query normalization) is applied, so that zero-mean and unit-variance scores are obtained per query. Then, if n different systems are fused, detections are aligned so that only those supported by n/2 or more systems are retained for further processing (this is known as majority voting validation). Let us consider one of such validated detections, corresponding to a query q; if a system A does not provide a score for it, we use instead the minimum score that A has output for q. The same value is assigned to missed detections and non-target trials. In this way, a complete set of scores is prepared, which besides the ground truth (target/non-target labels) can be used to discriminatively estimate a linear transformation that produces well-calibrated scores that can be linearly combined to get fused scores. Under this approach, the Bayes optimal threshold -given by the effective prior (0.0148 for this evaluation)-is applied. The BOSARIS toolkit [3] is used to estimate and apply the calibration/fusion models.
Tables 1 and 2 show the results (performance and processing resources) for GTTS systems in the required and extended conditions, respectively. All the experiments have been carried out on a 2× Xeon E5-2450 (×8 core, ×2 HT) @2.10GHz, 64GB, under Linux Fedora 3.3.4-5.fc17.x86 64. The indexing phase involves just applying BUT decoders to extract phone posterior features. ISF, SSF and PMU values have been computed as if all the computation had been performed sequentially in a single processor (see [6]). Calibration and fusion costs have been neglected.
The contrastive systems 2, 3 and 4 (c2, c3 and c4) use the BUT decoders for Czech, Hungarian and Russian, respectively. The contrastive system 1 (c1) uses the concatenation of phone posteriors from the three decoders as features (and the average of non-speech posteriors for SAD). The primary system (p), which is the fusion of the four contrastive systems, increases MTWV in 5 absolute points (15% relative) with regard to the best contrastive (c1). In all cases, calibration and fusion parameters have been estimated on the development set. Late submissions fixed a bug in the fusion script (which did not count missed detections), thus leading to better calibrated systems. Note also that a 15% relative MTWV increase (nearly 4 absolute points) is obtained by using multiple examples under the approach described above (system c2-late).