We consider the task of collaborative recommendation of photo-taking locations. We use datasets of geotagged photos. We map their locations to a location grid using a geohashing algorithm, resulting in a user location implicit feedback matrix. Our improvements relative to previous work are twofold. First, we create virtual ratings by spreading users' preferences to neighbouring grid locations. This makes the assumption that users have some preference for locations close to the ones in which they take their photos. These virtual ratings help overcome the discrete nature of the geohashing. Second, we normalize the implicit frequency-based ratings to a 1-5 scale using a method that has been found to be useful in music recommendation algorithms. We demonstrate the advantages of our approach with new experiments that show large increases in hit rate and related metrics.
The advent of digital cameras and smart phones has revolutionised photo-taking. We now create more multimedia content than ever before; the content is more heavily contextualised, e.g. with increasingly accurate data from phone sensors such GPS receivers; and sharing the content has never been easier nor more commonplace.
Web sites, such as Flickr and Facebook, where
multimedia content is shared, implicitly capture contextualised
personal preferences over the places that people like to create
content of this kind, i.e. the places where they take photos.
The preference data is a resource from which we can build
personalized and contextualized recommender systems [
In this paper, we study the task of recommending geolocations to users, based on the data found in photo-sharing sites. In other words, the items in our recommender system are geolocations. The recommender may help a user discover new places where they can enjoy good views or nice settings, suitable for photo-taking.
Collaborative recommendation of geolocations for
phototaking has been explored in recent work by Phan et al.[
In this paper, we, like Phan et al., are concerned with recommending locations. We use geohashing rather than Phan et al.'s cartographic hashing to map the places where users took the photos to buckets. Then we make two main contributions. First, we create virtual ratings by spreading users' preferences to neighbouring grid locations. This makes the assumption that users have some preference for locations close to the ones in which they take their photos. These virtual ratings help overcome the discrete nature of the geohashing. Second, we normalize the implicit frequency-based ratings to a 1-5 scale using a method that has been found to be useful in music recommendation algorithms. We evaluate the e ect of these two innovations separately and together using experiments that measure hit rate and related metrics, rather than RMSE.
In Section 2 we review the related work; in Section 3 we present our proposed method; and in Section 4 we give the experimental results. 2.
There is an amount of previous research in recommending
locations to users, e.g. [
There are other location recommenders that also work in
a coordinate space. For example, Liu et al. try to predict the
next location that a user will visit [
Shared photos have been used as a data source for
purposes such as POI detection [
The literature also contains descriptions of systems that
assist with photo composition, e.g. [
Our use of virtual ratings is similar in spirit to the
approach of fuzzy event modelling proposed by Hidasi and Tikk
[
In this section, we explain our proposed approach. The approach consists of geohashing, followed by the spreading of users' preferences by creating virtual ratings in neighbouring buckets, followed by the conversion of implicit feedback to 1-5 ratings. Finally, we use a collaborative recommendation algorithm on the resulting feedback matrix. 3.1
We do not use Phan et al.'s CASH method, preferring to use geohashing, which is more common. Both have the same e ect: they divide the surface of the Earth into a grid of rectangular cells (called bins in CASH and buckets in geohashing); a hash function takes in latitude and longitude and maps them to one of the cells of the grid. Geohashing maps latitude and longitude into a geohash key of up to 12characters. Coordinates that map to the same key are in the same cell of the grid (bucket). The scheme is hierarchical: pre xes of the hash key designate larger cells that include those designated by extensions of the pre x. The size of the pre xes is known as the precision. In this paper we take 7-character long pre xes. These designate buckets that at the Equator are 152.9 metres wide and 152.4 metres high.
It is interesting to note that Phan et al. run experiments where they vary the resolution of their hashing method. Altering the resolution, however, does not just a ect sparsity, it also alters the items (bins) that get recommended. Arguably, this should not be a parameter that one alters to minimise error. It is instead something one should x at a granularity that users nd useful for photo-taking. As we said, we x precision at 7, which give buckets that are about 150 150 metres, which we think is an appropriate size for photo-taking location recommendation. Had we used a precision of 6, buckets would be approximately 1km by 600 metres, which is clearly too large for useful recommendations. A case can be made for precision of 8 (about 40 by 20 metres) but anything higher is probably too small (e.g. precision of 9 recommends 5 by 5 metre locations).
It is important also to say that the size of the buckets, which we use to recommend where to take photos, is unrelated to the size of what might be photographed. From a bucket on the south bank of the River Thames, for example, a user might capture a panoramic shot of the north London skyline or she might zoom in on a pigeon eating a discarded hamburger. This emphasises the point too that recommending photo-taking buckets is not the same as recommending POIs. In the same example of a bucket on the south bank of the River Thames, there might be a POI in the same bucket (e.g. the London Eye) but the user might be taking photos of POIs in a di erent bucket in the distance (e.g. Big Ben) or may not be taking photos of speci c POIs at all (e.g. skylines and pigeons).
After hashing, we have an initial user location ratings matrix, where locations are buckets and ratings are based on frequencies. Figure 1 shows an example. Suppose a user u has taken six photos in six di erent locations. Suppose loc1, loc2 and loc5 are geohashed to the same bucket g1. The ratings matrix contains triples such as hu; g1; 3i, meaning that user u has taken three photos in bucket g1. 3.2
One problem with hashing to a rectangular grid is its discretization of coordinate space. In Figure 1, for example, taking a photo at loc2 is taken as positive feedback for that point in space and others near it. But the rating is recorded only for bucket g1. The geohashing results in us recording no positive feedback for the nearby points in g2.
Our solution to this problem is to create virtual ratings in the user location matrix by spreading the original frequencies to neighbouring buckets. First, we decide which buckets to spread to. We may spread to zero, one or more of the eight neighbouring buckets. We only spread from a bucket to a neighbour if the bucket contains a photo-taking event that is close enough to the neighbour. We calculate the geodesic distance between the coordinates of the photo-taking events in the bucket and the centre of the neighbour.1 Only if the minimum of these distances is smaller than a threshold value will we create a virtual rating. For example, in Figure 1, the rating for g1 will only be spread to g4 if the distance between loc1 and the centre of g4 (this being smaller than the distances from loc2 and loc5) is smaller than .
Next, we decide the value of the virtual rating. Its value is
a discounted version of the one that is being spread.
Following [
There is, however, the issue of how to aggregate ratings that `arrive' in a bucket from di erent sources. For such cases, we use the simple heuristic that the virtual rating is the maximum of the ratings arriving from di erent sources. For example, in Figure 1, bucket g4 receives two virtual ratings. One comes from g1: it is g1's rating (3) discounted by an amount based on the distance d14. The other comes from g5: it is g5's rating (1) discounted by an amount based on distance d34. The larger of these will be taken as the virtual rating for g4. Note that the same calculation is used even if g4 already contained a rating of its own: its new rating is the maximum of its original and the two discounted virtual ratings. Since the virtual ratings are discounted by an amount based on distance, only in exceptional cases will they replace an existing rating.
Spreading virtual ratings to neighbouring buckets does not, of course, enlarge what is being recommended. Recommendations continue to be made at the level of individual buckets. 3.3
Phan et al. normalize the frequency-based ratings to the
range (0; 1] [
Instead, we follow Celma's method [
Virtual ratings of music listening: to convert how often a user listens to a track into a 5-point scale. Unlike the kinds of 5-point explicit rating scales used in book and movie recommenders on the web, for Celma's normalized ratings, a rating of 1 does not necessarily mean that the user dislikes the item; rather, the fact that the item was listened to at all implies some level of positive feedback, but less enthusiastic positive feedback than that associated with higher points on the scale. It seems appropriate to use Celma's method for the implicit frequency-based ratings that we have in our photo-taking scenario. 3.4
At this point, we have a normalized ratings matrix. Our
goal, given a user and bucket for which the user has no
rating, is to predict the user's rating. For this, we use matrix
factorization to transform users and buckets into the same
latent factor space. We choose to use matrix factorization
since it is widely used for collaborative recommenders and
a form of matrix factorization was the best performing
approach in [
We collected the data used in this work from the photosharing website ickr.com by using its API. We searched for geotagged photos taken in London and Dublin in 2015 to create two datasets. After geohashing, we discarded users who had ratings for fewer than ve buckets. The nal dataset for London contains 112,671 photos taken by 978 unique users. Users have an average of 115 photos. The nal dataset for Dublin contains 54,082 photos taken by 1,567 users and users have an average of 34 photos. 4.2
We compared four con gurations of the recommender,
depending on whether ratings were normalized to (0; 1] (as in
[
We also compare against three baseline recommenders, POP H, POP ALL, and HOME, which we now describe. 4.2.1
Popularity-based recommenders
ones for which a recommender will be rewarded if it
recommends them in the experiments. In [
In fact, we have chosen to run all experiments twice: in one set of experiments, from the probe set we create a test set by retaining only test items where the user's normalised rating is 4 or 5; for the other set of experiments, we take all of the probe set as test set.
There are no virtual ratings in the test sets and, where di erent recommenders are compared (even ones that normalise to (0; 1]), they are compared using the same sets of test items.
For each test item, we randomly select 1000 other buckets for which the user has no rating. We predict the user's ratings for all 1001 buckets and then sort them by descending predicted rating. We then recommend the top-k (k = 10), which may or may not include the test item.
This means that experiments are being done on an item by item basis for each item in the test set, rather than on a user by user basis. We must de ne our metrics accordingly.
For each test item, we measure whether the test item is in the top-k or not. If it is, we call this a hit and record the total number of hits, H. From this, we calculate the hit rate (or recall):
HR =
H jTest j
We compare our methodology with a baseline recommender
that recommends the most popular items for which the user
has no rating. It is well-known that, in recommender
systems in general, recommending the most popular items can
be a highly competitive baseline [
Home location-based recommender
Using a Flickr dataset, Van Laere et al. conclude that a
user is more likely to take pictures in locations that are closer
to home [
Some Flickr users provide a textual description of their home location in their Flickr pro le. For the users in our datasets, we wrote a crawler to visit their pro les and obtain their home location descriptions, if given. (Of course, not all of these descriptions will be correct, and this may reduce the performance of this baseline recommender.) We convert the textual description to a geolocation (latitude and longitude).2 For the London dataset, we were able to obtain the home geolocations of 40.52% of the users; for the Dublin dataset, it was 39.5% of the users.
The baseline recommender, which we refer to as HOME, works as follows. For each user whose home location coordinates are known, we calculate the geodesic distance between their home location and the centres of the buckets (items). Then we recommend the closest buckets for which the user has no rating. For users whose home location coordinates are not known, we default to recommending the most popular buckets in the dataset, as POP ALL would do. 4.3
Phan et al. calculate the RMSE between the predicted and
actual ratings for the members of a test set. We chose to
base our experiment on a newer methodology, emphasising
recommendation over prediction [
We used 5-fold-cross-validation using 80% for training and 20% for a probe set. From the probe set, we construct a test set. The idea is that the test set will contain items from the probe set that the user liked, and therefore these are 2We do this by using the geopy python library, which uses the Google Maps V3 geocoder: https://developers.google. com/maps/documentation/geocoding/intro (1) (2) (3) where rank rui is the position of this test item in the top-k (1 rank rui k) or zero if this test item was not recommended in the top-k.
Although ARHR considers the positions of the hits,
neither ARHR nor HR gives information about the original
rating as well as the position. It is common to discount
this based on the logarithm of the rank. Hence, inspired
by discounted cumulative gain (e.g. [
When using virtual ratings, there is the parameter , which needs to be set for the experiments. With small values of , there are fewer virtual ratings than with larger values of . We tried values of 0 (which is the same as no virtual ratings), 75, 150, 225 and 300. In Figures 2 and 3, ADG =
1 X rruuii jTest j rraunik2rTuies6=t;0 log2(rankrui ) if rank rui = 1 if rank rui > 1 where Test is the test set.
We also calculate the average reciprocal hit-rank (ARHR), which in our setting we de ne as follows:
ARHR = 1
X
1 jTest j rraunik2rTuies6=t;0 rank rui we show the hit rate, average reciprocal hit-rank and average discounted gain for one of the system con gurations (5-VR) with varying values of for the London and Dublin datasets, respectively.
As can be seen, HR, ARHR and ADG tend to increase as increases but then, as becomes too big and so ratings are being spread too far, HR, ARHR and ADG level o or even fall. The Figures show that the most competitive value for for this system con guration for both datasets is 225.
The results for 1-VR (not shown) follow a similar pattern, but its most competitive value for is 300 for both datasets.
These are the values we use for in the results that we show in the next section. 4.5
We now compare the seven recommenders, i.e. the three baselines (POP H, POP ALL and HOME) and the four congurations of our recommender, which depend on the normalisation scheme and whether virtual ratings are used or not (1, 1-VR, 5 and 5-VR). As per the previous section, the two con gurations that use virtual ratings use their most competitive values for .
Figures 4, 5 and 6 show the hit rate, average reciprocal hit-rank and average discounted gain respectively.
The worst-performing system overall is the one designated
1, corresponding roughly to the system in [
The best-performing system is 5-VR | the one that combines both our innovations. It out-performs all recommenders on all metrics across both datasets.
There are also some other patterns in the results. For both datasets and all evaluation metrics, of the four non-baselines, system 1 is the worst; adding virtual ratings (1-VR) makes an improvement; scaling to 1-5 (system 5) is better again. But the best-performing con guration is 5-VR, where we scale to 1-5 and use virtual ratings.
Another pattern is that the baselines are more competitive on the Dublin dataset than the London dataset. They never out-perform system 5-VR, but on the Dublin dataset, POP H and POP ALL both out-perform systems 1, 1-VR and 5. The performance of the HOME baseline is more mixed.
For reference, Table 2 shows the values on which the Figures in this section are based.
In fact, as explained in Section 4.3, we chose to run two set of experiments, di ering in the way in which the test set was constructed from the probe set. In one case, all ratings from the probe set are placed into the test set. These are the experiments whose results were shown in the Figures and in Table 2. In the other case, only highly-rated items from the probe set are included in the test set. The results for this set of experiments are given in Table 3. Comparing the two tables, we see that numerically the results in the second set of experiments are higher. But, for the most part, the story about which systems out-perform each other remains the same. In particular, 5-VR remains far and away the best of the recommenders.
We presented an approach to recommending geolocations to users for photo-taking. We geohashed coordinates to cells in a rectangular grid, and used these as the items in an implicit feedback matrix. We investigated two innovations. One was to create virtual ratings in neighbouring cells. The other was to normalise ratings using a method developed for music recommenders. Our experiments, measuring hit rate, average reciprocal hit-rank and average discounted gain, showed that the two innovations together outperformed all other con gurations and popularity-based and home location-based baseline recommenders.
There are many avenues for future work. For a start, we can test on other datasets and on recommender algorithms other than SVD. There remains an open question about the precision of the geohashing. Here we are using precision of 7, resulting in buckets that are about 150 metres by 150 metres. The only way to determine whether this is the best choice or whether higher precision (smaller buckets) would be better is through a user trial. It may even be that precision should be personalised or adaptive in some other way, and this could be investigated in future work.
Finally, there is the opportunity to integrate other factors
into the work. Presently, we recommend only the
phototaking location. Date and time may also be important:
perhaps a location may o er better candidate subjects for
photos in certain seasons, on certain dates or at certain times of
day. Related to date and time are the photo-taking
conditions: a bucket may be a better location, e.g., when the sun
has risen, when the weather is not overcast or when the sun
is not glaring (although, of course, for some photographers
these di cult conditions provide opportunities for the
exercise of their photographic talents). Some users might also
want assistance with the choice of subject, camera settings
or composition. For this, it may be possible to integrate our
work with the kind of work done on the social camera [
This publication has emanated from research supported in part by a research grant from Science Foundation Ireland (SFI) under Grant Number SFI/12/RC/2289.