In this paper, we describe the LIA system proposed for the MediaEval 2013 Spoken Web Search task. This multilanguage task involves searching for an audio content query, in a database, with no training resources available. The participants must then nd locations of each given query term within a large database of untranscribed audio les. For this task, we propose to build a language-independent audio search system using an i-vector based approach [2].
The Spoken Web Search (SWS) task is characterized by two major di culties. Firstly, the reference set is composed of audio les coming from di erent languages, accents and acoustic conditions. Secondly, no transcription or language resources are provided. Systems should then be built as generic as possible to succeed in nding queries appearing in these multiple condition sources.
In this work, a language-independent audio search system
based on an i-vector approach is proposed. Inspired by the
success of i-vectors in speaker recognition [
Initially introduced for speaker recognition, i-vectors [
The i-vector extraction could be seen as a probabilistic compression process that reduces the dimensionality of speech super-vectors according to a linear-Gaussian model. The speech (of given speech recording) super-vector ms of concatenated Gaussian Mixture Model (GMM) means is projected in a low dimensionality space, named Total Variability space: ms = m + T xs (1) where m is the mean super-vector of Universal Background Model (UBM)1. T is a low rank matrix (M D R), where M is the number of Gaussians in the UBM and D is the cepstral feature size (39 in our case), which represents a basis of the reduced total variability space. T is named Total Variability matrix ; the components of xs are the total factors and they represent the coordinates of the speech recording in the reduced total variability space.
The proposed approach uses i-vectors to model speech segments. These short segments are considered as a language basic unit. Indeed, each le and each query are segmented in short segments of 20 frames. In our model, the segment super-vector m(seg) is modeled as follows: mseg = m + T x(seg) (2) 2.2
In this section, the di erent steps used to build the proposed language-independent audio search system are detailed.
In this rst step, the MFCCs (39-dimensional feature vectors) are computed for all the database audio les. Each vector represents 30 ms of Hamming windowed speech signal (the window is shifted every 10 ms).
This step consists of segmenting the database les in short segments of 20 frames. In this step, a sliding window of 200 ms with an o set of 100 ms is used in order to avoid information lost. 713,315 short segments are obtained after the segmentation of the 10,762 audio les. The same procedure is applied on the sets of queries (development and evaluation). The 505 evaluation queries are segmented in 1The UBM is a GMM that represents all the possible observations. It is sometimes also called the world model. 6,602 short segments (nearly the same for the development queries).
The i-vectors of each of the 713,315 segments of the database les (see step 1) are estimated based on the equation 2. An i-vector xseg of size 20 is then obtained for each segment. The UBM used (512 Gaussians) is estimated on all the database les. The Total Variability matrix T (dimension of 19; 968 20) is estimated using all segments of audio les.
In this step, the i-vectors for the queries (development and evaluation) are estimated using the equation 2. We used the same U BM and the Total Variability matrix T obtained in the last step. Finally, 6,602 i-vectors (size 20) for the evaluation queries are obtained (nearly the same for the development queries).
In order to identify the locations of each query term within the database audio les, an alignment is performed between the i-vectors of each query and the i-vectors of all database audio les using an adapted Dynamic Time Warping (DTW) algorithm.
The dissimilarity between two i-vectors is computed with the Mahalanobis distance2. The matrix used in the Mahalanobis distance is the total covariance matrix estimated on all the i-vectors of the database audio les.
In order to nd the query start, alignment can start at any point of the audio le but have to last at most two times the size of the query. The start and end times of a matching query are given by the start time of the rst ivector and the end time of the last i-vector of the alignment, respectively. The query score is the cost of the best path (more precisely, minus the cost). For each database le, the system searches the best costs of all queries although certain les do not contain any query. Only the n-best alignments for a document are kept (2, 3, 4, 6 or 8).
The proposed system is evaluated in the MediaEval 2013
SWS benchmark [
4)=(total duration of audio f iles total duration of queries)) = 6; 600=(71; 839 696) = 0.000132 4.
In this paper, we proposed a language-independent audio search system based on an i-vector approach. Although the results on the evaluation queries are poor, the encouraging results obtained on the development data show that the ivectors are an interesting and original unsupervised way to search audio content using an audio content query. In the future, we plan to investigate in details the mismatch between the development and the evaluation data performance. We will also explore the use of a Voice Activity Detection (VAD): it could help to discard silence sections that are contaminating audio queries.
This work was funded by the ContNomina project supported by the French National Research Agency (ANR) under contract ANR-12-BS02-0009.