We described Dublin City University (DCU)'s participation in the Search sub-task of the Search and Hyperlinking Task at MediaEval 2014. Exploratory experiments were carried out to investigate the utility of prosodic prominence features in the task of retrieving relevant video segments from a collection of BBC videos. Normalised acoustic correlates of loudness, pitch, and duration were incorporated in a standard TF-IDF weighting scheme to increase weights for terms that were prominent in speech. Prosodic models outperformed a text-based TF-IDF baseline on the training set but failed to surpass the baseline on the test set.
Increasing amounts of multimedia content are being produced and stored on a daily basis. In order to make this data useful, computer applications are required that facilitate search, browsing, and navigation through these large data collections. The MediaEval Search and Hyperlinking task seeks to contribute to addressing this problem.
In contrast with previous years where a known-item task
was examined, this year an ad-hoc search task was
introduced. The retrieval collection consists of an extension of
last year's collection, comprising 4021 hours of BBC TV
Broadcast content split into training and test sets of 1335
and 2686 hours respectively. For every video le in the
collection, the organizers provided human-generated subtitles,
three di erent automatic speech recognition (ASR)
transcripts (LIMSI/Vocapia, LIUM, and NST-She eld), prosodic
features, shot boundaries, visual concept detection output,
and additional metadata associated with each TV-show. The
training set includes 50 text queries while the test set
comprises 30 queries. More details about the data collection and
task evaluation metrics can be found in [
Previous research has demonstrated that prosodic
information is useful for a wide range of speech processing tasks [
Following Guinaudeau's method [
After the alignment was performed, minimum, maximum, mean, and standard deviation of loudness and pitch were computed for each word. These four statistics were normalised in order to be compared against other words spoken in di erent acoustic conditions. The nal objective was to calculate an acoustic score for each spoken word that represents how salient a word is relative to its surounding context. With this in mind, two di erent de nitions of surounding context for a word were then considered:
Context given by the words that belong to the same speech segment predicted by the ASR (seg).
Full length of document, this is, all the words spoken in the video (doc).
Finally, two normalisation functions were explored for normalising a feature fi over a context C: 1. Range: (fi 2. Z-score: (fi minC )=(maxC
C )= C .
minC ). 3.
Text transcripts were segmented into xed-time adjacent
(non-overlapping) segments of 90 seconds duration. Before
indexing, stop words from the standard Terrier list [
Subtitles NST-She eld
Normalisation Type Function Context
- range seg z-score doc - - range seg z-score doc - - range seg z-score doc - - range seg z-score doc - index. Note that due to stemming, multiple words can be mapped to the same stem. In these cases, acoustic feature vectors associated with each non-stemmed word ccurrence in a segment were treated as belonging to the same stem and thus were linked with this term in the inverted index.
Retrieval was performed using Terrier's standard
implementation of the vector space model (VSM) with a
modied TF-IDF weighting function that takes into account the
acoustic features from the inverted index when computing
term weights. The weight of a term t in a segment was
computed using Guinaudeau's harmonic mean [
ing and test sets were produced with di erent objectives in mind. This could be another reason why the models presented in this work seem to have over tted the training set.
In future work, an error analysis will be carried out in order to identify queries for which prosodic-based models could have outperformed the baseline. 5.
This work was supported by Science Foundation Ireland (Grant 12/CE/I2267) as part of the Centre for Global Intelligent Content CNGL II project at DCU.