A context-aware recommender system incorporates the knowledge of di erent contextual factors such as time or weather information to improve item suggestions made to a user. This requires the system to have a large knowledge base for inferring contextual information and enabling accurate and timely recommendations. We present a versatile approach for a context-aware recommender system in the tourism domain by crawling publicly available information from a variety of sources and learning the contextual popularity of points of interest based on a generalized check-in model. We have deployed a test instance of our system for the greater area of Munich and the German state of Bavaria. Analyzing the results from the o ine learning has led to interesting insights including when and in which weather conditions certain items are popular.
Recommender systems are a composition of software tools
and techniques that suggest items to users that are likely to
be interesting to them and relevant to their needs [
Context-aware recommenders enhance traditional
recommender systems by incorporating the knowledge of di erent
contextual factors - such as time or weather information
to further improve the item suggestions made to a user [
One major application area of context-aware recommender
systems is travel and tourism, where scenarios are signi
cantly more complicated than traditional user-product
matchings [
In our research, we propose, implement and evaluate a novel approach for a context-aware recommender system in the tourism domain by aggregating publicly available information from a variety of sources and learning the contextual popularity of POIs based on a generalized check-in model. We aggregate di erent types of data, including POIs, check-ins and contextual information to build a knowledge base and infer knowledge about the contextual popularity of items focussing on aspects of temporal and geographic context. The gained knowledge can also be utilized to mitigate cold-start problems when no or little information about the user is available.
In the following, we rst explore related work (Section 2), describe the context model, data sources and system design of our implementation (Section 3) and then present the process and results of analyzing the data (Section 4). Finally, Section 5 draws conclusions and discusses open future work directions. 2.
Context-aware recommender systems have been a topic
of growing research interest in the recent years and aim at
generating more relevant recommendations by adapting to
the speci c contextual situations of the user and the
recommended items (e.g., weather, temperature, season and
mood) [
Most current context-aware recommender systems work
in pull mode, i.e., the user has to explicitly make a request
(pull) for recommendations, possibly by entering
information about her preferences, needs and constraints. A new
generation of context-aware recommender systems, called
proactive recommender systems, are instead pushing
recommendations to users without their speci c request, when
they are in a contextual situation that the system considers
as suitable for the recommendations [
In another work, Dali et al. [
In another study, Pielot et al. [
Finally, Borr s et al. [
In order to create a versatile system that can gather relevant data for any geographic area and infer the contextual popularities of POIs, we rst discuss the context model that our work is based on. This leads to the features we need to extract from the data we crawl. We then discuss di erent data sources for items, check-ins and context and present a brief system design of both the backend implementation and a corresponding mobile client. 3.1
Our research is based on a context model for proactivity in
mobile recommender systems de ned by Worndl et.al. [
The context model de nes the data we need to aggregate and the features that should be extracted. In particular, we want to infer the temporal and geographic context of items. The user context is extracted directly in our mobile app prototype in a later stage. The social context is neglected in our prototype but could be integrated in our approach. Based on this context model, we focus on the following contextual features: weather condition, temperature, season, day of week, time of day and time of year. 3.2
The overall goal of this work to recommend POIs in a mobile tourist guide based on publicly available data sources. We therefore need to acquire and aggregate data in three categories: items (the POIs), check-ins (to determine the popularity of items) and context. 3.2.1
The POIs form the foundation of our system as the user's overall perception of the system rst and foremost depends on the quality and suitability of the recommended items. We therefore designed a general model for an item in our domain that can then be populated with data. Core data elds include: name, description, location, images, phone, rating, street, city, country. While the core elds should always be populated, each data source's crawler can de ne additional elds it wants to persists. Based on this model, we examined di erent, publicly accessible data sources: Foursquare, Yelp, Quermania, Facebook, Wikipedia, Open Street Maps. We designed and implemented crawlers for each of those sources and aggregated over 175,000 items for the German State of Bavaria. 3.2.2
Based on this knowledge base of 175,000 items, we want to infer the popularity in accordance to the context model. Our premise is that a place is popular if many people visit it. It follows that a place is popular in a certain context (e.g. on a sunny Saturday evening in summer) if many people visit it when this context condition applies.
To infer this knowledge, we must rst de ne a model for
determining how popular a place is at a given time or any
other context. We base this model on a generalized
checkin. In our model, a check-in can be any evidence that a
user visited a place at a certain time. This includes an
explicit check-in on Foursquare or Facebook, as well as implicit
check-ins by taking a picture or generating a GPS trace on
a smartphone. A check-in marks a singular point in time.
To make an educated judgement on how many people are
present at any point in time, we must further make an
assumption about the average duration of a visit. There has
been research on the activity duration of di erent activities
[
We looked at di erent publicly available data source to crawl check-ins from including Flickr, Twitter and Foursquare. Flickr is a photo sharing platform with a rich set of geotagged photos. A geo-tagged photo contains the latitude and longitude of the place it was taken as well as a level of accuracy of this information. In addition, a textual description is provided for most of the images. We used Flickr as the primary source of check-ins for rural areas, establishing a place-to-check-in mapping in a sparsely populated area which is more straightforward given the geo coordinates of both places. In our research, we have sampled about 249,000 images to be used as check-ins.
Twitter o ers a large number of tweets that can be associated with a place and hence counted as a check-in. There are two main features, that can be used for association: geotagged tweets and Twitter Places. Geo-tagged tweets carry a latitude and longitude and can be associated to places using geo-fencing. While this is a valid approach in rural areas, it is insu cient for cities due to the number of places being close to each other. Twitter places specify a speci c geographic place, such as a city or a restaurant and can be mapped to a POI directly. In our research, we have sampled 29m tweets in about eight weeks using Twitter's Streaming API.
Foursquare has an explicit check-in feature - similar to
Facebook - that would let users check-in at a place using a button
in their app. In addition, Foursquare samples the user's
location on the smartphone to proactively recommend POIs.
While this data seems promising, none of it is publicly
accessible and was therefore not used for our prototype. On the
contrast, some users link their Twitter account with their
Foursquare account resulting in each explicit check-in
triggering a public tweet. Following the approach proposed by
Melia and Segui [
Based on the aggregated knowledge base of items and check-ins we now want to add a third data source to enrich our data with context. We hereby focus primarily on dimensions of the geographic and temporal context. Given the timestamp of a single check-in we can infer all attributes of the temporal context using a static calendar library. We therefore use java.util.Calendar to infer season, day of week, time of day and time of year. To infer the weather condition and temperature of check-ins we added Wunderground's Weather History API that can be used to provide the weather for a place at any given time in the past. Using these two sources, we were able to infer the temporal and geographic context for all check-ins. 3.3
We implemented our prototype in Scala using the Play! framework. Scala is a language that is executed on top of the Java Virtual Machine and is fully interoperable with Java. The language combines object-oriented programming with functional programming which makes operations on arrays and lists easy to implement. On top, all Java libraries are usable in Scala which is a major bene t. The Play! framework o ers a modern web MVC architecture with a simple control ow and the advantage of using templates in the views. We have combined this with Slick - a functional relational mapper - to have a persistent database layer based on a performance-tuned MySQL instance. Slick o ers a clean way of accessing and ltering persistent data sets using Scala's functional API.
Data aggregation is either done through a REST API (if available) or through one of our crawlers. In the course of this research, we have created multiple crawlers and spiders on top of JSoup to extract knowledge from publicly available sources. We used Amazon AWS resources to carry out parts of these tasks with on-demand resources, while maintaining only one dedicated server at all times. We used SQS, a distributed pipe service, to communicate between the di erent servers. 3.4
We have also designed, developed and tested a user
interface concept to investigate how to communicate
contextual information about recommended items to the user in a
mobile tourist guide [
To receive early feedback for our concept, we evaluated
our system in a user study with 14 subjects. The
participants were asked to test the application for two weeks and
then complete a survey about their experience with the user
interface elements. The study results indicated that the
popularity inference and graphs provide bene ts for users. Users
stated that popularity graphs assisted them while deciding
which place to visit. More detailed results of this
preliminary study can be found in [
The focus in this paper is not on the user interface but on how to analyze the collected data and infer insights that can then be integrated in a mobile tourist guide. We present our course of action and the gained results of the o ine learning in the next section.
To infer basic popularities for di erent conditions we
follow the data analytics process proposed by Runkler [
To follow this process, we de ne an overall research question and subquestions that we seek to answer.
Overall research question: how can we learn (infer) the popularity of POIs for a context-aware recommender system? Subquestion 1: how can POIs from di erent sources be matched and duplicates be eliminated? Subquestion 2: how can POIs be associated with checkins? Subquestion 3: given a set of check-ins for a POI, how can we infer the popularity under di erent temporal conditions? Subquestion 4: given a set of check-ins for a POI, how can we infer the popularity under di erent geographic conditions? Subquestion 5: given a set of check-ins for a POI and the base popularities, which algorithms are suitable for making a compound decision/recommendation? In the following, we present our approaches and results on these tasks according to the subquestions we are trying to answer. 4.1
We have added di erent POI sources to our system, including Foursquare, Yelp and Quermania. Especially Foursquare and Yelp provide a lot of overlapping data, since both have restaurants and bars in their database. This leaves us with the problem of identifying and merging co-referent POIs, as we do not want to show or recommend the same POI multiple times.
Merging co-referent POIs has been extensively studied
and there are multiple approaches to the problem, e.g.,
using a fuzzy set and probability theory [
Associating POIs with check-ins is a non-trivial task, given that the data comes from entirely di erent sources. Depending on the source of the POI and the check-in, we have identi ed several ways of creating an association: 4.2.1
Flickr and Twitter both provide data records that have coordinates, a timestamp and some textual data like the tweet's content or the Flickr image's caption. Associating these records with a POI presents a hard problem that can be tackled in many di erent ways:
Simple Geofencing: The computationally easiest option is to use a rectangular geofence around a POI and associate each check-in record within this area with the POI. This works well for exposed POIs (i.e., satellite POIs that have no other POIs around them), but is inapplicable to the majority of POIs in cities including restaurants and bars.
Advanced Geofencing: To improve both the FPR (false positive rate) and FNR (false negative rate) of the simple geofencing approach, we propose a more complex way of setting up the geofence. Depending on the POI's type, a more complex geofence structure can range from a smaller circle (suitable, e.g., for bars or restaurants in cities) to a polygon following the shape of ski slopes or trails.
Clustering: can be done in a supervised, unsupervised or semi-supervised fashion. A supervised clustering approach would assume that we have a couple of check-ins per POI where we have obtained evidence that they belong to this POI. Other nearby check-ins could be associated using algorithms like KNN (see Evidence-based Clustering below). An unsupervised approach could use algorithms like DBSCAN to discover clusters in unclassi ed data. Once the algorithm has discovered clusters, we could use di erent techniques to associate POIs with clusters.
Evidence-based Clustering: As mentioned with simple clustering, one approach could be to use supervised or semisupervised clustering for association. To start this algorithm we would need a base dataset of check-ins that are associated with POIs. One approach to generate such a dataset would be to use evidence from the metadata. Most tweets/Flickr descriptions contain the topic of the tweet or image such as "Neuschwanstein". Using simple word-matching or more complex NLP techniques we would obtain a base set of check-ins to use for clustering and association.
We use Flickr and Twitter check-ins for exposed sights to deliver a proof of concept for our pipeline and model and could therefore use simple geofencing to associate check-ins with POIs (by a square whose size depends on the POI type). Our approach has linked 280,983 check-ins with 177 sights. While this approach was su cient for our use case, future work could lie in exploring other association techniques to make using these check-in sources viable for more densely populated areas. 4.2.2
As brie y outlined before, Melia and Segui [
This approach is promising, as it established a de nitive relation between a check-in and a POI (i.e., there are no false-positives). We have aggregated a mixed data set of tweets containing both geo-located tweets and tweets linked to public Foursquare and Swarm check-ins using the streaming API over eight weeks. The dataset has 29m tweets in total, 2,9m of which we have successfully linked to a Foursquare or Swarm POI.
Our prototype only incorporates POIs in the German State of Bavaria. We could therefore only use a small subset of the 2.9m tweets that link to a POI in our region. We could match tweets and POIs at a rate of 8 check-ins (tweets) per 1,000 POIs. This number is relatively low compared to the e ort it took to obtain the data using the pipeline we outlined earlier. While this was not an ideal t four our use case, the approach can be a better choice in regions with higher Foursquare adoption (such as New York or San Francisco). 4.3
Until this point, we have crawled POIs, check-ins and context and associated POIs with check-ins. The next step is to lter the dataset to increase its quality, decide on sampling for context parameters and perform normalization if needed.
Filtering: For the qualitative evaluation and our mobile app prototype we focus on exposed sights out of our large POI dataset and have identi ed 176 sights with a high POI to check-in matching having both a low FNR and FPR. Furthermore, we lter out POIs that have less then 400 checkins (on average each POI has 1570 check-ins) to retain a good accuracy for all contextual popularities. All contextual popularity groups partition the remaining check-ins into a maximum of 7 groups yielding 57 check-ins on average per group. This ltering gives us a set of 114 sights in the greater area of Munich and the German State of Bavaria that we use for further processing.
Sampling: The next step to aggregate the popularity is to de ne what dimensions should be explored and if a dimension is continuous or discrete. We hereby explore temporal and geographic dimensions separately. Discrete variables are easy to handle when it comes to inferring the popularity, as they can be represented by a xed number of buckets (e.g., seven for Day of Week ) and assigning check-ins to buckets is trivial. Continuous variables however, are more di cult as questions like "How popular is this POI at 10am?" can not be holistically answered using sampling and buckets, since when settling on a xed bucket size (e.g., half an hour) the question "How popular is this POI at 10:11am?" can not be answered accurately. When decreasing the bucket-size, buckets will only have a few check-ins as a check-in only marks a speci c timestamp making the popularity impossible to infer.
We therefore explored related work on activity duration
[
We regard the Gaussian modeling as future work and base our presence interval on activity durations without augmenting Gaussians. We introduce feltTemperature as a discretization of the continuous temperature as this simpli es inference and makes our predictions easier to understand for users. Table 1 shows our set of inferred contextual variables for both the geographic and temporal context.
dayOfWeek timeOfDay
season feltTemperature weatherCondition
D D
fMon,...,Sung
[0,23] fSpring,Summer,Fall,Winterg fHot,Warm,Pleasant,
Mild,Cold,Freezingg fSunny,Cloud,Rainy,Snowyg
Normalization: Figure 2 shows an aggregation of all of our check-ins on a weekly basis using the presence window approach based on a presence window of 1 hour. One can clearly see from the graph, that about twice as much checkins are made during weekends, than there are on a weekday. We have therefore thought about normalizing the data as a whole to have the same relative amount of check-ins per discrete bucket (e.g. Mon-Fri).
After investigating our data in close detail, we decided against normalization as users tend to visit more POIs on the weekends/evenings when they spend leisure time. We therefore anticipate, that the weekend cli s (as well as some other data patterns) are correct and bene cial for the sake of popularity inference. 4.4
4.4.1
To e ectively visualize our data, we created multiple analysis tools based on our server-side software. For e ective analysis of the check-in data we build a tool to visualize different features of geo-based check-ins without an association to any POI. Our tool, as depicted in Figure 3, is able to apply a geofence to lter check-ins and compare the selected area to a baseline of all known check-ins which makes it easy to spot interesting patterns. Figure 3 shows all check-ins on a weekly level for the northern end of Starnberger See. This is a good example for the e ectiveness of our tool, showing that this part of the lake is popular on Friday afternoon as well as on the weekend.
After an initial analysis using these tools, we started to model the popularity inference. For this purpose, we created di erent views, such as the one depicted in Figure 4, to visualize the inference outcome. Our view shows the probabilities (popularities) under di erent temporal and geographic conditions highlighting them with a gradient in red color such that one can easily spot patterns in the data. 4.4.2
Until this point, our analysis and visualizations yielded promising patterns. We now take a probabilistic approach to infer popularities for di erent contextual situations. The outcome we seek is the popularity P of a POI given a context C: p(P jC) (1)
When thinking about context and di erent dimensions as described before it would be most bene cial to split up C into temporal and geographic dimensions and their respective parameters (temperature t, weather condition wc, season s, weekday wd, etc.). This yields probabilities of the form:
p(P jt); p(P jwc); p(P js); p(P jwd)
Having these base popularities would allow for compound calculations yielding the probability for a visit V to a certain POI given a situation with xed contextual parameters:
The concept of a visit is that given that a user will certainly visit a POI, what is the probability that the visit will happen at the context C.
There are di erent approaches to model the statistical dependence of context factors like weather condition and temperature (e.g., it does not snow when having 20 C) including Bayesian networks. While a full Bayesian model can increase accuracy, it requires a high degree of domain knowledge about the data and all parameters that is hard to obtain. We therefore use summation to infer the plain popularities for each context parameter: p(P jwd = M onday; V = W alhalla) = p(wd = M ondayjV = W alhalla)
= p(wd = M ondayjV = W alhalla)
P
CheckinsF orP OIOnMonday
P
CheckinsF orP OI (2) (3) (4) (5) (6) (7)
This approach yields a probabilistic distribution for each POI and each context dimension as depicted in Figure 4 which lets one judge under which conditions a place is popular. We can use prediction and the Bayes' theorem to make compound recommendations. 4.4.3
With prediction or recommendation, we seek to answer the question "Where should I go (given the current timestamp and weather)?". With respect to our model, this would mean: what is the probability (or score) that I should choose a POI for my visit V given the inferred popularities of all
For each POI we add the popularity for a speci c set of context dimension (weather condition wc, temperature t, day d, season s, hour h) that we want to predict a score for. Furthermore, we multiply each dimension by a weighting factor to make the model exible. While we used static weights, future work could include learning of through online of o ine learning techniques.
While this model works well, it does not respect the different number of check-ins between POIs. Thus a POI with only 400 check-ins might outperform a POI with 4,000 checkins, as the popularity values are better, while - in absolute visitor numbers - the second POI outperforms the rst. This issue is addressed by an approach using Bayes' theorem.
Bayes' theorem can be used to inverse a dependent probability. Using summation to obtain the base probabilities as outlined in the previous section yields probabilities in the form of:
p(wd = M ondayjV ) In other words, this means: "given that I visit a POI, what is the probability I would visit it on Monday?" (or: "what is this POI's Monday popularity"). When recommending items, we would like to answer questions of "It is Monday, which POI should I visit?":
p(V jwd = M onday) So for a concrete POI (Walhalla) we can use Bayes' theorem to get the probability for a visit given our limited set of POIs and the premise that the user will visit one of these: p(V = W alhallajwd = M onday) = p(wd = M ondayjV = W alhalla)p(V = W alhalla) p(wd = M onday) (12) All of the parameters from this model can be retrieved from our dataset: p(V = W alhalla) =
p(wd = M ondayjV = W alhalla) (as obtained in the previous section)
P
CheckInsAtW alhalla
P
AllCheckIns p(wd = M onday) =
P
AllCheckInsOnMonday
P
AllCheckIns (10) (11) (13) (14) (15) when being relatively unpopular and therefore used weighted additive scoring.
Using either one of those strategies leaves us with a probability (popularity) for each POI in the database, which can be used for ranking in combination with a standard weighted additive scoring approach for recommending a POI based on the di erent context factors. 4.5
We discussed di erent types of evaluations for this part of our research with peers from the eld of machine learning. While machine learning and data analytics applications are usually evaluated using cross-validation or any other sort of quantitative o ine evaluation, we did not see a t for these types of evaluations given our data and inference schema. We therefore decided to rst evaluate our research by qualitatively assessing inferred popularities for selected results in a case study and discussing interesting ndings and patterns. The ultimate goal is to integrate the results about item popularities into the mobile application (see Subsection 3.4) and conduct a large-scale user study to get real feedback.
The inferred seasonal popularity performed best among all our inferred parameters. An excerpt of the POI data with seasonal popularity is shown in Table 2. While cities like Bad Tolz or Nurnberg are equally popular throughout the year, other sights peak in certain seasons. Kehlsteinhaus for instance - a tea house built for the Third Reich government that has been turned into an exhibition - is closed during spring and winter (November - April), as the road can not be maintained as soon as there is snowfall. It can be easily seen that this shutdown persists in the data making the sight less popular during winter and spring. Other places for outdoor activities show a clear tendency towards the warmer periods of the year, including Walchensee (being popular during Summer and Fall), Starnberger See (Summer) and Konigssee (Summer and Fall).
Bad Tolz Nurnberg Walchensee Kehlsteinhaus Starnberger See Kloster Andechs Konigssee
The weather condition overall has a clear tendency towards sunny weather, with more then 90% of the dataset having a sunny popularity of 0.5 or higher. Nonetheless, there are still some patterns to get indications if the inference technique is correct. While cities like Munich are within our average (mostly sunny), places for outdoor activities like Starnberger See, Andechs or Almbachklamm have a notably higher sunny popularity. Table 3 shows an excerpt from inferred weather condition popularities.
We found, that using a Bayesian approach
disproportionally favors POIs that have a high number of check-ins even
In this work, we have designed and implemented a data
analytics process to infer the popularity of POIs for a
contextaware recommender system using publicly accessible data
from various data sources. We have elaborated di erent
approaches on individual subtasks, e.g., applying probabilistic
modeling to learn the popularity for POIs in di erent
contextual situations such as weather condition, day of week
and time of day. We qualitatively evaluated our approach
in a case study and also implemented a corresponding mobile
application with a brief preliminary user study [
In addition to the explained results, we discovered other useful purposes for the dataset and the inferred information. For example, to determine and visualize popular areas on a map. This could be useful for visitors to nd spots for taking iconic pictures or maybe nd a less-crowded spot and avoid the most popular areas. To do so, we ltered our data by rst selecting the check-ins associated with a POI and then applying another lter to eliminate check-ins with an accuracy worse then 10 meters. Then this ltered dataset is fed into a heatmap algorithm for visualization using Google Maps' heatmaps extension. Figure 5 shows popular (photo) spots around Neuschwanstein Castle as an example.
Future work includes integrating the item popularities into the mobile tourist guide application and proactively recommending POIs based on the current user context. We then plan to conduct a larger user study to investigate whether this approach leads to useful and timely recommendations from a user's perspective in a real setting. Another area for improvement is not to solely rely on explicit user check-ins but also utilize tracking data from smartphones.