In this paper, we describe our approach to the Search and Hyperlinking task at the MediaEval 2013 benchmark. This task focuses on video retrieval and linking in the context of a large and rich dataset provided by the BBC. Our approach makes use of one of three types of audio transcripts, enriched with Named Entities. To compute similarity, we adapt the Jaccard metric to use Named Entities. This results in an unsupervised and computationally inexpensive way of searching and linking multimedia content.
The Search and Hyperlinking task at MediaEval 2013 [
For our approach, we only make use of the transcripts
provided with the dataset. For each video, three types of
transcripts are provided: human-generated subtitles, and two
ASR transcripts, one provided by LIUM [
These research activities were funded by Ghent University, iMinds, the IWT Flanders, the FWO-Flanders, and the European Union. 2.
Our approach consists of two main phases: ingestion and run execution.
As it is time consuming, the ingestion phase is performed
before the runs are executed. It has three steps: content
representation, segmentation, and enrichment. As content
representation for the videos, we use one of the three
provided transcripts, either the subtitles, LIUM transcripts, or
LIMSI transcripts, selected at runtime. For the
segmentation step, the timing information of the transcripts is
used to divide the videos into segments. Due to the
success of time-based segmentation in the 2012 benchmark [
The execution phase consists of two steps: similarity calculation and result selection. In case of the search task, the similarity between the queries and the segments in the dataset is calculated, whereas in case of the linking task, this is done between the anchor segments and the other segments in the dataset. To calculate the similarity, we opt for a completely unsupervised similarity metric. This way, no computationally expensive training step is necessary, that would have to be repeated upon expansion of the dataset, as would be the case when using a supervised similarity metric. In our case, we use the Jaccard metric, applied to NEs. We calculate the similarity between two enriched documents (be it segments, videos, or queries) A and B as follows: Sim(A; B) = jfe : e 2 E(A) \ E(B)gj ; jfe : e 2 E(A) [ E(B)gj (1) where E(A) and E(B) denote the sets of extracted NEs from document A and B, respectively. Note that no NEs could be extracted from some of the queries provided for the search task, due to their short length. When this is the case, we revert to a fallback mechanism, using a slightly altered metric. The fallback mechanism consists of creating a set of keywords for both the query and segments using a naive keyword extraction algorithm. This algorithm extracts all 1http://spotlight.dbpedia.org/ distinct words in the query or segment as keywords, with all stop words2 removed. We then employ the same calculation as in Equation 1, with the di erence that E(A) and E(B) now represent the sets of keywords. For each segment of length L, the optimal result segment length is determined, by maximizing the similarity score for x consecutive segments, with x = 1; 2; :::; W for a maximum window size W and maximum segment length Lmax (in our case, W = 4, and Lmax = 2 minutes). Finally, the N segments with the highest similarity score are selected and ranked, to be returned as results of the run (in case of our runs for the search task, N = 500; in case of the linking task, N = 20).
There are other approaches found in literature that make
use of NEs, sometimes dubbed as \concepts", for the purpose
of measuring similarity between documents. These concepts
are mostly used to determine weighting schemes, such as
CF-IDF [
We submitted ve runs in total, three for the search task,
and two for the linking task. Our three runs for the search
task were performed using the approach described in Sect. 2,
with each run using one of the three transcript forms:
subtitles (S), LIUM (U), or LIMSI (I). Before the runs, the
queries were enriched with NEs using the same NER service
as used for the segments. Then, the similarity between the
queries and all the segments in the dataset was computed
as described in Sect. 2. In Table 1, we present the mean
reciprocal rank (MRR), mean generalized average precision
(mGAP), and mean average segment precision (MASP) of
the three search runs. For more information about these
evaluation metrics, we refer to the task overview paper [
S
MRR 0.0322 0.0546 0.149 mGAP 0.0222 0.0322 0.0906
MASP 0.0268 0.0515 0.123
For the linking task, we submitted two runs, each making use of the subtitles, since these lead to the best result in the search task. The rst run (A) was performed using only the segment that contained the anchor itself as input, whereas the second run (C) also made use of the context of surrounding segments, de ned by the user who chose the anchor. The runs were evaluated by human users, resulting in precision at rank x 2 f5; 10; 20g (Px), and mean average precision (MAP). The results are shown in Table 2.
C MAP 0.0375 0.0459 0.3200 0.3867 P10 0.2800 0.3500 P20 0.1667 0.2050
When we compare the results of both tasks with those of
our (partially) supervised late fusion approach, submitted to
the benchmark of 2012 [
In future work, we aim to experiment further with unsupervised similarity measures, adapted to work with NEs. Also, the in uence of the NER service used needs to be evaluated. Other, more accurate NER services than DBpedia Spotlight exist. However, their free versions are mostly limited in number of requests, making them less suitable for datasets of this magnitude, without signi cant costs. Application of our approach to multilingual content also remains an important challenge we aim to address. We also aim to exploit more of the semantic features of NEs, taking advantage of the similarity between individual concepts. 6.