Recommendations within a network do a ect, and get affected by, the information ow and the social connections within the network. Thus, designing a network-centric recommender system requires understanding people's preferences, their social connections, as well as the characteristics of the network they inhabit. This creates a major challenge in research on network-centric recommendations|exploring questions around networks and recommendation is hard because they invariably depend on the interaction between more than one user. We describe a research platform that we have built that helps us answer network-centric research questions. We present an overview of the system and demonstrate its usefulness through an example study involving directed suggestions between pairs of participants. As a useful side-e ect, it is also helping us collect data about people's preferences in social networks.
Recent research on recommendation using social networks
has taken two main approaches|augmenting collaborative
ltering with social data [
One of the reasons for mixed results with network-aware
approaches may be that people's explicit social connections
have little to do with their interests in a particular domain.
Thus, in domains where social connections overlap with user
interest, social recommendation may be useful, in others, not
so much [
However, we argue that there is more to explicit social
networks that o ine measures of recommender performance
may not capture. For instance, studies with Facebook and
Google users have found that showing the name of a
particular friend with a recommendation (music or news) can alter
a user's perception of it. [
In addition, a user's preferences are not static; they are
continuously being in uenced by their network. The structure
and properties of a network, as a whole, are also a ected by
the connections between people and their activities within
the network [
Framed this way, a challenge of network-centric recommendation is that it makes designing and evaluating systems hard, because they invariably depend on interactions between more than one user. This paper describes how we can use PopCore1, a research platform we have built, to explore questions around networks and recommendation. As a network-centric system built on top of Facebook's social network, PopCore serves as a research tool to support live evaluations of network-centric recommendations, and collection of user reactions and feedback in a real network setting. It is also a functional recommender system in its own right, much like the MovieLens system2.
Figure 1 shows a screenshot of PopCore, which we rst
proposed two years ago in this workshop [
For each recommendation, a user may Like it on Facebook, rate it, recommend it to some friends or add it to a personal queue (Figure 2). When a user clicks on the Recommend button for an item, he can choose one or more of his friends to recommend the item to. For convenience, we also o er a
list of recommended recipients who PopCore predicts might be interested in the item (based on the similarity between the item and the user's friends).
Visualizations, such as the item cloud of most popular items Liked by a user's friends and updates of recent activity on the app (Figure 1), help create increased network awareness for the user. User controls on the right, which allow a user to tailor recommended items based on their popularity, social closeness or similarity of the people connected with those items, also help users nd interesting content and navigate their friends' interests.
We now describe how we are using the system as a test bed for research. In addition to the main website, we have a created a separate labs website3. It runs on the same core infrastructure, but has custom code that allows us to run experiments. Participants are asked to sign up and give permission for each experiment separately.
The core infrastructure of PopCore supports an ego-centric
view of the network for each user, collecting Likes and
network information about the user and his friends that can
be used for a variety of studies. In our rst experiment,
we showed users a variety of di erent recommendation
algorithms, some of which used past Likes of the users' friends.
Users were asked to rate and Like items and react to the idea
of network-centric recommendation [
In the study described above, participants did not know
the social nature of the recommendations. Subsequently, we
have used the platform to study how presenting a social
explanation along with a recommendation can a ect user
ratings for music [
Besides live experiments, data collected for the above
studies helps us reason about the properties of networks and
how they might a ect recommendation. In one such study,
we compared o ine performance of algorithms using
preferences of friends or the whole network, and found that
algorithms using just the friends' information give comparable
results to those using the full network [
Till now, we have described how PopCore can be a useful platform for experiments involving a user and social data. However, the system is designed to support participation and interaction between multiple users. We now describe an ongoing experiment as an example of how PopCore can be a useful tool for conducting experiments with more than one participant.
We consider the practise of sharing items between people, more commonly known as \word of mouth". Intuitively, suggesting an item to another person may involve two factors: having an opinion about the item, and having an under
standing of the receiver's preferences. Understanding the processes behind directed suggestions, speci cally around item and receiver selection, can help design recommendation systems that support such suggestions.
One of our rst goals is to simply compare the recommendation quality of directed suggestions versus algorithmic recommendations. In the following paragraphs, we present how the PopCore platform can be useful for investigating this question.
Experiment Design: Previous research on helping people
share news items in RSS feeds suggests that manual
recommendations to friends can be useful [
A simple way to compare the quality of recommendations would be to analyze logs of user ratings in PopCore and compare the performance of directed suggestions and algorithmic recommendations. However, this comparison will depend greatly on the particular recommender algorithm being used. One way to control for the recommendation algorithm is to allow only those directed suggestions that are also recommended by the algorithm we are comparing against. Then, among the algorithmically recommended items, we would be able to compare the ratings between items that were manually suggested and items that were not. Such a comparison would point to the quality of manual and algorithmic recommendations, as perceived by the user. For ease of logistics, we design a movie recommendation experiment involving pairs of participants. The system generates its top 10 recommendations for each participant in a pair. It then randomly shu es the 20 recommendations for a pair into a combined list, and shows the combined list to each participant. Similar to the main PopCore interface, participants are free to rate them or suggest them to their partner. We show this combined list to participants so that both of them rate and suggest from the same set of recommendations. This is done to ensure that both partners can choose to start the study at di erent times and still rate each other's potential suggestions in a single session. In addition, comparing ratings for the same item from both sender and receiver could tell us whether people tend to recommend items that they themselves rate highly, or those that may be relevant based on the other's preferences.
Experiment Flow: We run the study using the labs version of PopCore. In the rst stage, a participant invites one of his friends as his partner. Once the other participant accepts the invitation, she enters the experiment and rates/suggests 20 items. The rst participant then gets an email and he can go ahead and rate/suggest the same 20 items. During the experiment, we highlight the other participant's name whenever a user wishes to make a directed suggestion (Figure 3), in order to make the directed suggestions easy to send out. Thus, at the end of the experiment, we receive ratings for the items from both participants and a list of the items that they shared to one another. Participants are also presented with a questionnaire that asks about how close they are to the other participant, and the reasons they receive or send recommendations in general. Note that participants have no way of detecting which items were suggested to them by their partner. Thus, the ratings can be assumed to pertain to the quality of the recommendation, rather than the social connection between the sender and the recipient.
Initial Results: We are currently in the process of collecting data. We are recruiting participants from a mix of college student pools and Amazon mechanical turk. From an analysis of the initial data on 27 pairs, we nd that items which were manually suggested have a higher average rating than those that were algorithmically recommended. This supports the notion that manual recommendations are useful, and in some cases, better than an algorithmic recommendation. However, it may also be the case that people tend to recommend highly popular items which are more likely to get higher ratings. We hope to use o ine data to corroborate these ndings.
In addition, we nd that the average rating for manual recommendations is higher for the senders than receivers. This result suggests that users may be sharing items that they like more frequently than those which they think can be relevant for the recipient.
We have presented PopCore, a research platform for conducting user experiments with recommendations in social networks. Our goal for this paper is to show how the system has helped us conduct experiments that require social data and/or active participation of users embedded in a network. We envision it as an open platform for research, for which we would like to discuss potential ways of collaboration at the workshop. For instance, one of the ways is to allow other researchers to setup experiments on PopCore. In addition, we also hope to share the data we are collecting, keeping in mind the potential challenges around user privacy in generating those datasets.