This paper describes our Georeferencing approaches, experiments, and results at the MediaEval 2014 Placing Task evaluation. The task consists of predicting the most probable geographical coordinates of Flickr images and videos using its visual, audio and metadata associated features. Our approaches used only Flickr users textual metadata annotations and tagsets. We used four approaches for this task: 1) an approach based on Geographical Knowledge Bases (GeoKB), 2) the Hiemstra Language Model (HLM) approach with Re-Ranking, 3) a combination of the GeoKB and the HLM (GeoFusion). 4) a combination of the GeoFusion with a HLM model derived from the English Wikipedia georeferenced pages. The HLM approach with Re-Ranking showed the best performance within 10m to 1km distances. The GeoFusion approaches achieved the best results within the margin of errors from 10km to 5000km.
The MediaEval 2014 Placing task requires that
participants use systems that automatically assign geographical
coordinates (latitude and longitude) to Flickr photos and
videos using one or more of the following data: Flickr
metadata, visual content, audio content, and social information
(see [
place names, stopwords lists, and an English Dictionary.
The system uses the following rules from Toponym
Disambiguation techniques [
2) Hiemstra Language Model (HLM) with Re-Ranking.
This approach uses the Terrier2 Information Retrieval (IR)
software (version 3.0) with the HLM weighting model [
The indexing process uses a multilingual stopwords list to lter the tokens that are indexed. The following metadata elds (lowercased) from the photos/videos were used for the query: User tags, Title and Description. A Re-Ranking process is applied after the IR process. For each topic their rst 1000 retrieved coordinates pairs from the IR software are used. From them we selected the subset of coordinates pairs with a score equal or greater than the two-thirds (66.66%) of the score of the coordinates pair ranked in rst position. Then for each geographical coordinates pair of the subset we sum its associated score (provided by the IR software) and the score of their neighbours at a threshold distance (e.g. 100km). Then we select the one with the maximum weighted sum.
3) GeoFusion: Hiemstra Language Model with ReRanking and GeoKB. This approach is applied by combining the results of the GeoKB approach and the IR approach with Re-Ranking. From the GeoKB system are selected the predicted coordinates that come only from the heuristics 1, 2 and 3 (avoiding predictions from the population heuristics rules). When the GeoKB rules (applied in priority order: 1, 2, and 3) do not match then the predictions are selected from the HLM approach with Re-Ranking 2Terrier. http://terrier.org 4) GeoFusion+GeoWiki: GeoFusion combined with a HLM model of Georeferenced Wikipedia pages. This is the only improvement with respect to the system used at MediaEval 2011. This approach uses a set of 857,574 Wikipedia georeferenced pages3 that were indexed with Terrier. The coordinates of the top ranked georeferenced Wikipedia page are used as a prediction. The predictions from the georeferenced Wikipedia based HLM model are used only in case that the HLM model with Re-Ranking based on the Training data gives an score lower than 7.0. This threshold was found empirically training with the MediaEval 2011 test set. The system uses the coordinates of one of the most photographied places in the world as a prediction when the approaches cannot give a prediction.
We designed a set of four experiments for the MediaEval 2014 Placing Task (Main Task) test set of 510,000 Flickr photos and videos (see results in Figure 1 and Table 1): 1. The experiment run1 used the HLM approach with Re-Ranking up to 100 km and the MediaEval 2014 training set metadata as a training data. From a set of 5,050,000 photos and videos of the MediaEval 2014 training set, a set of 3,057,718 coordinates pairs with related metadata info were created as textual documents and then indexed with Terrier. 2. The experiment run3 used the GeoKB approach. 3. The experiment run4 used the GeoFusion approach with the MediaEval training corpora. 4. The experiment run5 used the GeoFusion approach with the MediaEval training corpora in combination with the English Wikipedia georeferenced pages HLM model.
We used four approaches at MediaEval 2014 Placing Task. The GeoFusion approaches achieved the best results in the experiments clearly outperforming the other approaches. These approaches achieve the best results because combine high precision rules based on Toponym Disambiguation heuristics and predictions that come from an HLM models. The GeoKB rules used in the GeoFusion approach achieved 81.17% of accuracy (131,207 of 161,628 photos/videos) predicting up to 100km. The most di cult cases for prediction with our textual based approach are the ones with few textual 3http://de.wikipedia.org/wiki/Wikipedia:WikiProjekt\_Georeferenzierung/Hauptseite/ Wikipedia-World/en 30 % 20 % 10 % information and tags. In this evaluation we tried an approach that uses sometimes the English Wikipedia Georefenced pages to handle these cases. The GeoFusion+GeoWiki approach (that uses an HLM model of English Wikipedia georeferenced pages) does not generally o ers better performance than the original GeoFusion approach. This approach only improved very slightly the results for estimations at 10km. The HLM approach with Re-Ranking obtained the best results in the 10m to 1km range because the model takes some bene ts of relating non-geographical descriptive keywords and place names appearing in the geographical coordinates' associated metadata.
This work has been supported by the Spanish Research Department (SKATER Project: TIN2012-38584-C06-01). TALP Research Center is recognized as a Quality Research Group (2014 SGR 1338) by AGAUR, the Research Department of the Catalan Government. 5.