This paper provides a description of the MediaEval 2011 Placing Task. The task requires participants to automatically assign latitude and longitude coordinates to each of the provided test videos. This kind of geographical location tag, or geotag, helps users localise videos, allowing their media to be anchored to real world locations. Currently, however, most videos online are not labelled with this kind of data. This task encourages participants to nd innovative ways of doing this labelling automatically. The data comes from Flickr|an example of a photo sharing website that allows users to both encode their photos and videos with geotags, as well as use them when searching and browsing. This paper describes the task, the data sets provided and how the individual participants results are evaluated.
This task invites participants to propose new and creative approaches to tackling the problem of automatic annotation of video with geotags. These tags are usually added in one of two ways: by the photo device (e.g. camera or camera-equipped mobile phone) or manually by the user. An increasing number of device are becoming available that can automatically encode geotags, using the Global Position System, mobile cell towers or look-up of the coordinates of local Wi-Fi networks. Users are also becoming more aware of the value of adding such data manually, as shown by the increase in photo management software and websites that allows users to annotate, browse and search according to location (e.g. Flickr, Apple's iPhoto and Aperture, Google Picasa WebAlbums).
This work was supported by the European Commission under contract FP7-248984 GLOCAL. However, newly uploaded digital media and videos in particular, with any form of geographical coordinates, are still relatively rare compared to the total quantity uploaded. There is also a signi cant amount of data that has already been uploaded that does not currently have geotags.
This task challenges participants to develop techniques to
automatically annotate videos using their visual content and
some selected, associated textual metadata. In particular,
we wish to see those taking part extend and improve upon
the work of previous tasks at MediaEval and elsewhere in
the community [
The data set is an extension of the MediaEval 2010
Placing Task data set [
Development data is the combination of the development and test data from the MediaEval 2010 Placing Task. The two sets are pooled to form the 2011 development set. We include as much metadata as is publicly accessible to make available to participants a variety of information sources for use when predicting locations. This includes the title, tags (labelled Keywords in the provided metadata les), description and comments. We also include information about the user who uploaded the videos and about his/her contacts, his/her favourite labelled images and the list of all videos she/he has uploaded in the past.
It should be emphasised that the task requires the participants to predict the latitude and longitude for each video. The prediction of the names of locations or other geographic context information is outside the scope of this task. The development set comes with the ground truth values for each video. This information is contained in the metadata in the eld <Location>. Frames are extracted at 4 second intervals from the videos and saved as individual JPEG-format images, using the freely available mpeg1 tool. 2.1.2 Flickr images For development purposes, we distribute metadata for 3,185,258 Flickr photos uniformly sampled from all parts of the world, using geographic bounding boxes of various sizes via the Flickr API(http://www.flickr.com/services/api/). Whilst the images themselves are not distributed in this task, they are publicly accessible on Flickr (if they have not been removed since the data set was gathered) and the provided metadata contains links to the source images.
From these images, their existing metadata is extracted. Most, but not all, photos have textual tags. All photos have geotags of at least region level accuracy. The accuracy attribute encodes at which zoom level the uploader used when placing the photo on a map. There are 16 zoom levels and hence 16 accuracy levels (e.g., 3 - country level, 6 - region level, 12 - city level, 16 - street level).
While these images and their metadata are potentially helpful for development purposes, the evaluation test set, however, only includes videos.
We also generated visual feature descriptors for the extracted
video keyframes and training images, using the open source
library LIRE [
Colour Layout
The Scalable Colour Edge Histogram and Colour Layout
features are implemented as speci ed in the MPEG-7 schema [
The geo-coordinates associated with the Flickr videos will be used as the ground truth. Since these do not always serve to precisely pinpoint the location of a video, the evaluation will be carried out at each of a series of widening circles: 1km, 10km, 100km, 1000km, 10000km. If a reported location is found within a given circle radius, it is counted as correctly localised. The accuracy over each circle will be reported. The orthodromic and Euclidean distances between the ground truth coordinates and those reported by participants will also be calculated. We are also interested in the issue of videos that have been uploaded by an uploader who was unseen in the development (i.e., training) data. In order to examine this issue, we calculate a second set of scores over the part of the test data containing only unseen uploaders.
Participants may submit between three and ve runs. They can make use of image metadata and audio and visual features, as well as external resources, depending on the run. A minimum of one run that uses only audio/visual features is required. The other two required runs allow for the free use of the provided data (but no other), with either the option of using a gazetteer or not. Participants may submit an optional additional run that uses a gazetteer, as well as a optional run that allows for the crawling of additional material from outside of the provided data (the general run). Participants are not allowed to re- nd the provided videos on-line and use actual geotags (or other related data) for preparing their runs. This is to ensure that participants help contribute to a realistic and sensible benchmark in which all test videos as \unseen". The participants are also asked to not crawl Flickr for any additional videos or images and use only those provided in the data sets (with exception made for the optional general run).