Abstract

We describe the participation of the USEMP team in the Placing Task at MediaEval 2014. We submitted four textual runs which are inspired by CEA LIST's 2013 participation. Our entries are based on probabilistic place modeling but also exploit machine tag and/or user modeling. The best results were obtained when all these types of information are combined. The accuracy of automatic at 1km reaches 0:235 when using only training data provided by organizers and 0:441 with the use of external data.

1. INTRODUCTION

The goal of the task is to produce location estimates for a test set of 500,000 images and videos using a set of approximately ve million geotagged images and videos and their metadata for training. A full description of the challenge and of the associated dataset is provided in [ 1 ]. Our runs were implemented using, for a large part, methods described in CEA LIST's participation at Placing Task 2013 [ 2 ]. For this reason, after a short presentation of the methods, runs and obtained results, we focus on failure analysis. 2.1

METHOD DESCRIPTION

Probabilistic location models

Language models are successfully introduced in [ 3 ] as an alternative to gazetteer-based geolocation and were progressively improved in following years. Test photos can be placed anywhere in the physical world and the training data provided by the organizers is insu cient in order to build robust probabilistic models. To verify the assumption that better results are obtained with the use of more data, we exploited: (1) all geotagged metadata from the YFCC dataset 1, after removing all test items and (2) an additional set of 90 million geotagged metadata from Flickr.

Similar to last year [ 2 ], the surface of the earth was split in (nearly) rectangular cells of size 0.01 of latitude and longitude degree (approximately 1km2 size). User counts were used instead of tag counts in order to mitigate the in uence of bulk tagging. Both titles and tags were taken into account and are referred to as tags hereafter. Put simply, we 1http://webscope.sandbox.yahoo.com/ catalog.php?datatype=i&did=67 computed the probability of a tag in a cell by dividing its user count in that cell by its total user counts in all cells. Given a test item, we simply summed-up contributions of individual tags to nd the most probable cell for that item. Finally, the photo was placed at the center of the cell. 2.2

Machine tag modeling

The authors of [ 4 ] show that machine tags can improve automatic geotagging quality. In [ 2 ] we propose a machine tag processing method which models only machine tags which are strongly associated to locations. (i.e. Foursquare, Lastfm and Upcoming entries) and we exploited it this year. 2.3

User modeling

If images do not have associated tags or if these tags are not geographically discriminant, placing photos with probabilistic models is likely to fail. To overcome this problem, we exploited a simple user modeling technique [ 2 ], which computes the most probable cell of a user. Only photos which are at least 24 hours away from any of the user's test set images were exploited to reduce the risk of learning from test data. We downloaded up to 500 geotagged images per user in order to determine her most probable cell. 2.4

Fusion

We propose a late fusion scheme which is empirically derived from tests with a validation dataset. Since they are associated to precise locations or geolocated events, processed machine tags are very reliable and were used in priority. If there were no machine tags, location models were exploited to predict the most probable location of a set of tags. Finally, if there were no tags available or if the prediction score was below a threshold, the photo was placed in the most probable cell of the user who uploaded it. The threshold for replacing location models with user models was empirically determined on the validation dataset. We exploited user models for the 30% of test images which had the lowest placing scores. 3.

RUNS

We submitted the following runs: RU N1 - exploited location models and machine tags from training data provided by the organizers; RU N3 - combined location models and machine tags from the entire geotagged YFCC dataset, after excluding test items; RU N4 exploited tags and user models; RU N5 - exploited YFCC location models, machine tags and user models. We present the performance of the submitted runs in Table 1. The best results were obtained when com

P@X km 1 10 0.235 0.408 0.428 0.582 0.418 0.597 0.441 0.613 bining all types of available information. As expected, the largest contribution was due to location models. The large gap between RU N1 and the others con rms that the use of supplementary training data is very bene cial. The di erence of precision at close range (P@0.1) between RU N3 and RU N4 con rms that machine tags are very useful for precise geolocation. Inversely, if larger errors are admitted, user models become more useful than machine tags. The combination of these types of cues in RU N5 gives the best performance for all precision ranges. The results obtained this year are in the same range as those we reported in 2013 [ 2 ], con rming thus that our geolocation pipeline has consistent behavior over di erent datasets.

FAILURE ANALYSIS

In addition to the submitted runs, we tested other con gurations which gave lower results and brie y describe them here. We notably tried a combination of location models and gazetteer information in order to give a privileged role to toponyms such as administrative division names (i.e. countries, regions, cities). The addition of the gazetteer gave lower results compared to the sole use of location models. This negative result could be explained by the strong ambiguity which characterizes the geographic domain. As we mentioned, we also tried to add a dataset of 90 million geotagged metadata to the YFCC full training data. Contrarily to existing literature [ 4, 2 ], the use of this supplementary dataset actually degraded the overall quality of results. This negative result might indicate that probabilistic models reach saturation when too much metadata are available.

In Figures 1 and 2, we present a visualization of geotagging performance for RU N1 and RU N5 and the performance di erence between the two runs is clearly re ected . Geotagging is precise in most European regions and worse for the other regions. Low performances can be easily explained by sparse data for Africa, Asia or South America. However, the imprecision is also high for the United States, the region of the world which concentrates the largest number of geotagged images. In this case, poor geotagging could be due to a very high ambiguity of place names. For instance, there are dozens of places called London or Paris in the US. If there is not enough disambiguation information associated to them in annotations, photos tagged with these toponyms will be placed in Europe. 5.

FUTURE WORK

Due to lack of time, we did not submit a visual run this year. While visual geotagging still lags well behind textual geotagging, it would be interesting to explore if it is possible to predict coordinates accurately at least for visually distinctive objects such as Points of Interest. Regarding text models, we would like to investigate in more depth why adding more data from outside YFCC degrades performance. Equally interesting, it would be interesting to investigate ways to select reliable annotations before computing location models.

ACKNOWLEDGMENT 6. 7.

This work is supported by the USEMP FP7 project, partly funded by the EC under contract number 611596.

References

[1] J. Choi and al. The placing task: A large-scale geo-estimation challenge for social-media videos and images . In Proc. of GeoMM'14.

[2] A. Popescu . Cea list's participation at mediaeval 2013 placing task . In MediaEval , 2013 .

[3] P. Serdyukov and al. Placing ickr photos on a map . In Proc. of SIGIR 2009 .

[4] M. Trevisiol and al. Retrieving geo -location of videos with a divide & conquer hierarchical multimodal approach . In ICMR, pages 1{8 , 2013 .