This paper presents our approach and results in the Search and Anchoring in Video Archives task at MediaEval 2015. The Search part aims at returning a ranked list of video segments that are relevant to a textual user query. The Anchoring part focuses on the automatic selection of video segments, from a list of videos, that can be used as anchors to encourage further exploration within the archive. Our approach consists in structuring each video into a hierarchy of topically focused fragments, to extract salient segments in the videos at di erent levels of details with precise jump-in points for them. These segments will be leveraged both to answer the queries and to create anchor segments, relying on content based analysis and comparisons. The algorithm deriving the hierarchical structure relies on the burstiness phenomenon in word occurrences which gives an advantage over the classical bag-of-words model.
This paper presents the participation of IRISA at the
MediaEval 2015 Search and Anchoring in Video Archives
task [
Our approach consists in structuring each video as a
hierarchy of topically focused segments using the algorithm
proposed in [
The aim of our approach is rst to nd precise jump-in
points to the salient segments in the videos, at various
levels of details. These segments are obtained by applying the
algorithm proposed in [
Subsections 2.1{2.3 detail the following: 2.1, the generation of potential anchor and query-response segments; 2.2, the selection of the top 20 segments for each query; 2.3, the ranking of the anchor segments. 2.1
Each video in the test collection is partitioned into a hierarchy of topically focused fragments with the automatic segmentation algorithm HTFF, which is domain independent, needs no a priori information and has proven to o er a good representation of the information contained in videos. It can be applied on any text-like data. For the search subtask we are provided also with the visual query, i.e., visual concept words. For the search sub-task LIMSI transcripts and the visual concepts detected for each keyframe in the video were used; While for the anchor detection sub-task LIMSI and manual transcripts were used. For applying the algorithm, data in the transcripts are rst lemmatized and only nouns, non modal verbs and adjectives are kept.
The core of HTFF is Kleinberg's algorithm [
A cosine similarity measure is computed between each query and the content of the segments previously retrieved. This measure is computed with segments from all levels in the hierarchy and the ones for which higher similarity is obtained compared to the others will be ranked higher. In this setting, short, focused and highly similar segments are favored. This procedure is done both for textual and visual query independently. 2.3
After having the list of salient segments for every video
for which anchors need to be extracted from, we compute a
cohesion measure to rank these fragments. The measure is
a probabilistic one where lexical cohesion for a segment Si
is computed using a Laplace law as in [
ni + k j=1 where ni is the number of word occurrences in Si, fi(wji ) is the number of occurrences of the word wji in segment Si and k is the number of words in V. The quantity C(Si) increases when words are repeated and decreases consistently when they are di erent. Using HTFF for anchor detection does not ensure any number of anchor segments to be found for a video. Therefore, some videos might have more or less anchors proposed than others. This is realistic, since the number of anchors that can be found in a video depends on the information contained.
For the search sub-task, 30 test set queries were de ned. The top 10 results for each query were evaluated for each method, using crowd-sourcing technologies. Our results for the search sub-task are reported in Table 1. LIMSI denotes the system using LIMSI automatic transcripts and textual query, while Visual denotes the system relying on visual concepts and visual query. The best results are obtained with the LIMSI system. Analyzing the list of all the segments proposed by participants, it can be observed that with our approach the segments proposed are shorter in duration. Figure 1 illustrates the duration of the segments that were judged relevant or not with both our systems (LIMSI and
P 5 0.34 0.12
Visual) compared to those proposed by other participants. The segments we proposed are on average less than half the size of the segments proposed by other participants. This was detrimental to our approach. Some of the short segments, proposed with our methods, are judged not relevant. While long segments which cover these short segments are judged relevant. However, many of our short segment do not overlap with longer segments proposed by others, so in the end they remain judged as not relevant.
For the anchor sub-task, a list of 33 videos was de ned, for which anchors had to be proposed. The top-25 ranks for each video and each method were judged by crowd-sourcing using Amazon Mechanical Turk workers who gave their opinion on these segments taken from the context of the videos. Precision, recall and Mean Reciprocal Rank (MRR) measures have been used. The results obtained for both our systems LIMSI (using LIMSI automatic transcripts) and Manual (using subtitles) are reported in Table 2. The best results were obtained when relying on automatic transcripts. 4.
The results obtained on both sub-tasks show that while for anchor detection short segments are a good idea, for the search sub-task, assessors seem to need more context to judge a segment relevant. For future work on the search subtask we consider selecting larger segments from a higher level in the hierarchy (i.e., coarse grain). Additionally, combining visual and textual bursts could improve the results. For the anchor detection task, di erent ways to rank the segments could be considered, favoring segments which contain named entities or visual bursts.