This paper describes the work done by the JRS team for the linking sub-task. We submitted eight pairs of runs: four with di erent textual resources only, two using reranking based on visual similarity, and two using concept detection results. Each of the pairs contains of one run using the anchor segment only, and one using a longer context segment. The results show higher variance between anchors than for the 2013 task, also the di erences between runs using di erent textual resources are more salient. The use of the context does not generally improve results, and visual reranking provides small improvements.
The MediaEval 2014 Search and Hyperlinking of
Television Content Task addresses the scenario of performing
search in a video collection (search sub-task) and
subsequent exploration of related video segments (hyperlinking
sub-task). This paper describes the work done by the JRS
team for the linking sub-task. Details on the task and the
data set can be found in [
For the linking sub-task, we combine textual/metadata similarity and visual similarity. The textual/metadata similarity is based on matching terms and named entities, and provides a basic set of result segments. In some runs, visual similarity based on local descriptors is used for reranking. In the following, we brie y summarise the approach.
The textual/metadata based approach uses the automatic speech recognition (ASR) transcript or subtitles and the metadata about the broadcast (using title, description and short synopsis of episodes). All these textual resources are preprocessed by removing punctuation, normalizing capitalization and removing stop words and very short words (less than three characters). We then select a basic set of terms T = Ta [ Tm, which are the words Ta from the anchor and Tm from the metadata, that are found in DBpedia1. Some runs use the results of concept detection, treating the annotated concepts Tc like terms extracted from the text of the segment. In this case, the set of terms is de ned as T = Ta [ Tm [ Tc. For the ASR transcript or subtitles, we then broaden the set of terms and select speci c classes. As a rst step, we add synonyms for the terms in T from WordNet2, obtaining a set ST . We then select a set of connected entities CT for the terms in T from FreeBase3. For the subset of terms Tg T , which FreeBase identi es as related to a geographic location, we also add the set of connected geographic entities GTg from GeoNames4. Thus the set of terms used for matching is T = T [ ST [ CT [ GTg .
For matching two segments, we match the terms related to these segments with di erent weights: w(t) = wo; t 2 T; w(t) = wg; t 2 GTg ; w(t) = ws; t 2 ST [ CT ; with ws < wg < wo: (1) For multiple occurrences K in a segment, the weights of each occurrence decrease, with the total weight de ned as w(t) = PK b k=1(1=k)w(t). For a pair of video segments (v1; v2) the similarity is determined as Pt2T (v1)\T (v2) w(t), with T (vi) being the extended set of terms of segment vi.
For initial text-based matching, the videos have been segmented into segments of equal lengths of 20 seconds. In the experiments, we cut the lists at a normalized similarity score of 0.1, keeping at most 500 result items.
For visual matching we use VLAT [
We submitted in total eight pairs of runs, each
containing one using only the exact anchor segment, and one using
the anchor plus context item. This decision is based on the
conclusions from the linking sub-task at MediaEval 2013,
where the use of a longer segment including context signi
cantly improved the results. As no anchor item segment is
2wordnet.princeton.edu
3www.freebase.com
4www.geonames.org
0,5
0,4
0,3
0,2
0,1
0
de ned in the task input data in 2014, we created segments
by adding three minutes before and after anchor, as this was
shown in [
For the runs that use the same method as in 2013 (apart from necessary changes for parsing metadata) the overall scores are clearly lower, and the variance of the results is much higher. This indicates that the anchors used this year might be more challenging, and more diverse. Using context segments does only slightly improve results, but the impact depends very much on the anchor. For example, for anchor 25 we obtain signi cantly better results when only using the anchor, while for anchor 47 results are much better when using the context. There are also clearer differences between di erent text resources than in 2013, with manual subtitles clearly providing best results, LIMSI and NST/She eld have comparable results, and LIUM resulting in lower results. Using visual reranking provides in most cases a slight improvement (this is in line with the results from 2013). Visual concepts alone result in a performance that is an order of magnitude lower than using text results, combining them with text slightly lowers the performance for the anchor only runs, and signi cantly for runs using the context segment. An overview of the scores of some selected runs is shown in Figure 1.
Comparable approaches yield overall lower results than in 2013, and there is higher variance in the queries. This includes also the usefulness of context segments for the anchors. Visual information makes small contributions to the overall scores, however, visual concepts need to be treated specially, as their con dence is still low.
The authors thank Michal Lokaj for his contributions to the implementation. The research leading to these results has received funding from the European Union's Seventh Framework Programme (FP7/2007-2013) under grant agreement n 610370, \ICoSOLE { Immersive Coverage of Spatially Outspread Live Events" (http://www.icosole.eu/). 6.