In this paper we present our current experience of aggregating user data from various Social Web applications and outline several key challenges in this area. The work is based on a concrete use case: reusing activity streams to determine a viewer's interests and generating television programme recommendations from these interests. Three system components are used to realise this goal: (1) an intelligent remote control: iZapper for capturing viewer activities in a cross-context television environment; (2) a backend: BeanCounter for aggregation of viewer activities from the iZapper and from di erent social web applications; and (3) a recommendation engine: iTube for recommending relevant television programmes . The focus of the paper is the BeanCounter as the rst step to apply Social Web data for viewer and context modelling on the Web. This is work in progress of the NoTube project4.
is also duplicated across the various social sites. The user is left with multiple user pro les, which are not or weakly related (syntactically only) to each other.
Maintaining rich pro les of a wide variety of content (in various formats and areas) is critical for providing adequate personalisation services. Recommendation services are a typical example of applying such rich user pro les. However, most approaches today are still limited to simple recommendations (\Take a look at this programme"), similarity-based recommendations (\if you watch this programme, you will probably like this other one in the same category") or collaborative ltering (such as Amazon's \Customers who bought this item also bought..."). Services for television recommendations, such as 4IP's \Test Tube Telly"5, tend to be restricted to using data from one system only. An advanced mechanism for social recommendations will now need to consider a completely di erent landscape, with di erent pro les of users, user groups and audience segments. Opening and sharing of pro les and attention data between di erent players in the market creates a di erent environment - an open social graph where a wider user base can potentially help to improve the quality of recommendations and reduce the costs of moderation and spam ltering.
In the NoTube project [3] we develop a exible end-to-end architecture, based on semantic technologies, for personalised creation, distribution and consumption of television content. The project takes a user-centric approach to investigate fundamental aspects of consumers' content-customisation needs, interaction requirements and entertainment wishes, which will shape the future of \television" in all its new forms. In this paper we focus on the rst part of this project - combining multiple heterogeneous sources of data, with the addition of semantics, in order to create machine-readable pro les for users. These can later be used to drive better, more focused and more personalised television and other media recommendation services in a personalised, service-based EPG (Electronic Programme Guide)[4]. NoTube aims to overcome a number of limitations in current EPGs. Imagine how to recommend and search for programmes that are: (a) non- ction, (b) produced between 1989 and 1995, (c) involve locations in Eastern Europe. In current EPGs (a) is resolvable, while (b) is not, but it is possible if the programme information has been properly indexed along a given timeline; (c) also requires appropriate metadata, and a sophisticated knowledge model that allows to represent and reason about part/whole relations between places, countries and regions (e.g., So a is in Bulgaria; Bulgaria is in Eastern Europe). In the project we hope to gain insights into end-user control, user understanding of aggregation of data about them, privacy-preserving architectures, and constraints on reuse of data as data of this kind is both potentially privacy invasive and also valuable.
In this paper we describe the design and implementation aspects of the BeanCounter as the user data collecting component, aiming to illustrate its potential application in the Social Web domain. The main design rationale is to provide a exible and extensible architecture that exposes robust, scalable and reliable services to handle di erent kind of responses of di erent social applications plat
forms. A set of APIs allows for modelling the targeted responses to gather them, represent them with a set of suitable RDF vocabularies, and integrate them with other pulled information in a fully transparent way. Thus, we outline a loosely coupled architecture based on service-speci c adaptors (tubelet ) and application servers (modelet ), which allow for the selection of data source and RDF vocabulary and the generation of RDF-ised user data, integrated with data coming from other adaptors. We aim to create a set of language-independent, RESTful APIs that allow for writing adaptors, analysers, data management and enrichment components to di erent user data services. An important requirement is maintaining a transparent privacy policy useful to users with di erent levels of technical skills.
In the following sections we report on our work on the BeanCounter with usage scenarios, alignment approaches, design and implementation details of the BeanCounter implementation, and lessons learned. In these lessons we outline some of the challenges related to realising not only the data collection component, but the whole work ow for achieving reuse of user data from Social Web. 2 2.1
Bob has accounts at several social websites: Facebook, Twitter, YouTube, and Last.fm. He regularly uses NoTube `favourites' feature. For each website he has a separate identity and has indicated a basic set of personal information and interests. Each application also carries his (partial) user pro le, with his interests related to that application, e.g., music preferences or travel history. There is limited integration of such user pro les, typically not under user's control and lacking transparency of how data is interpreted in di erent applications. It is di cult for Bob to nd out what each system knows about him and how this knowledge is derived. Moreover, he cannot easily nd out his interests and personal statistics based on what he watches, listens to or consumes on di erent devices (e.g., interactive television, online TV guide, Net ix, YouTube). Using the BeanCounter, Bob can: { log into the BeanCounter using his openID (or create an account). { link a few of his accounts and devices to the BeanCounter, e.g., his Twitter and YouTube accounts. He could also link an iZapper (a combined EPG, remote, and context capturing device) to his account. After linking, the service-speci c adaptors (`tubelets') pull the data, convert it to an activity stream in RDF using service-speci c algorithms, and store it in a triple store. { view some statistics about himself . Bob can immediately see some interesting information about himself in the BeanCounter web Interface (see Figure 1). A machine-readable pro le can also be created. { edit the result, e.g., delete all the data or remove sources. He can decide which part of the pro le to make public, if any. The nal result is a machinereadable pro le that Bob uses to describe himself on the Semantic Web. Jane is privacy-conscious, technically minded and able to do some programming. She has the latest television-PVR that outputs data using the XMPP protocol; her friends have similar devices. She uses an iZapper to indicate programmes she watches, records or likes and dislikes. For example, Jane can:
Download and run her own version of the BeanCounter and attach her BeanCounter to her and her friends' PVR accounts, with their permission. In this way, she can make queries over the combined set of interests to see if she can make some recommendations about what to watch, by querying the dataset to see what is mostly commonly watched and then querying schedules to lter what's on next week. 2.3
George is a developer who wants to use the BeanCounter for tracking attention data from YouTube. Suppose an instance of the BeanCounter is currently deployed and is running on beancounter.asemantics.com, but is pulling data only from Twitter. George wants to track YouTube favourites as well. He studies the response (Atom feed) from the YouTube API and writes couple of Java classes to represent this data. He chooses an appropriate authentication mechanism (e.g., http Basic Auth, or OAuth). Then he needs to select the appropriate RDF vocabulary to represent this user data. FOAF is suitable to represent the static part of the YouTube pro le (e.g., the property foaf:interest links users to the tags associated with every YouTube favourite video. All he needs is to add a minimal set of annotation to his Java classes stating this. Once the lines of code are written, he uses the Management Interface beancounter.asemantics.com/manager and adds the YouTube Tubelet to the BeanCounter. The tubelet is immediately available to be used without restarting the whole service.
Consider now Chris, who is a developer at OpenCalais annotation service 6 and wants to enrich identi.ca tweets. The goal is to build a Identi.ca Tubelet that pulls the Identi.ca tweets, annotates them using the Opencalais and stores the resulting triples. Similarly to the previous scenario, Chris rst needs to make a new tubelet. Then, he writes a small Java class called a Pipe, to indicate that Opencalais service is to be used to process the text of the tweet, and nally, he binds the new tubelet to that pipe. Thus, the output of this tubelet will be processed by this pipe and then stored (see Figure 2). The same scenario can be realised with other annotation services, such as KIM annotation service7.
Another example is Anna, a BBC producer, who wants to nd out whether people liked the new episode of `Torchwood'. Suppose the BBC has an instance of the BeanCounter running and aggregates and anonymises data using the public APIs from Twitter and Facebook combined with statistics from iPlayer. Anna writes custom queries for Torchwood and a custom analyser that evaluates whether each activity data item was positive or negative. With the `aggregation'
feature - over all user data in the system - she creates custom queries to nd out what the overall opinions were and when people tended to watch it. 3
More and more data gets published in RDF[8] most notably as linked data[6], [5]: generic knowledge such as DBpedia8, domain data such as instances from the BBC Programmes Ontology9, linguistic data such as W3C Wordnet10, service data like instances of WSMO-lite11, or user data like FOAF12 instances or MySpace metadata13. This knowledge can be accessed in various ways[9]: SPARQL endpoints, Java APIs, and REST-style Web-services. Programmers combine these sources to create migration services for other programmers or end-user (mashup) applications. In developing the BeanCounter we have experienced ve key aspects relevant to the enrichment of the user attention data: { Selection of resources: The growth of the Semantic data cloud allows us to integrate more external data sources relevant to the NoTube domain. The selection of the right sources is a non-trivial challenge. First, we need to list candidate sources that possibly could contribute to our user-scenarios. Second, for those sources we need to estimate the quality in terms of completeness, reasoning complexity, errors and stability. Third, we need to determine the e ort needed to align the schemas to connect the source with the other NoTube vocabularies. { Creation of alignments: The growth of the Semantic data cloud allows more and more interesting alignments for the NoTube domain between the di erent data sources in the cloud. For example, in one of the case studies we are working on making alignments in SKOS to aligning the Last.fm music categories with programme data described in the BBC Programmes ontology. { Creation of connections: The linked data cloud contains commonly used URIs representing entities suitable for reuse, for example Dbpedia concepts. Where possible we reuse URIs in this way to create a more connected graph.
The challenge is to nd and easily reuse the relevant URIs { Usage of the alignments and connections: When the alignments are created, we cannot assume that all data that we want to align will be in one single SPARQL repository. For this we need to develop alignment services that themselves generate RDF on request or have another interface (e.g., REST-style). For example, a type of alignment that gets special attention in NoTube because of non-English content provided in the case-studies: multi-lingual aspects related to mapping Korean, Italian, Dutch and German
10 http://www.w3.org/TR/wordnet-rdf/ 11 http://www.wsmo.org/ns/wsmo-lite/ 12 http://xmlns.com/foaf/spec/ 13 http://grasstunes.net/ontology/myspace/myspace.html content to English (and back). For example, we are working on mapping the Dutch Cornetto `Wordnet' to the W3C Wordnet, to obtain more (English) information for topics annotated originally in Dutch. { Domain dynamics: The world is dynamic, so also the metadata describing this world, such as user pro le data, programmes and similar. It is the challenge to get the alignments as quickly as possible and to inform the interested parties. For this a publish/subscribe mechanism where people can place a `listener' on the topics of interest would make life easier. Current research such as Jqbus (a SPARQL service over the XMPP protocol) is ongoing to tackle this challenge.
The BeanCounter is a user data aggregation component: it aggregates data about a user from Web sources (e.g., Facebook, Last.FM, Twitter) and produces a machine-readable interest pro le for this user. The BeanCounter produces also a human-readable version of this pro le by emphasising on the transparency and privacy-preserving aspects, as well as on the adequate and appealing presentation of the information. However, in the context of this paper, we focus on its input to semantic recommendation services. In the prototype version of the overall NoTube architecture (see Figure 3) the BeanCounter is accompanied by a controlling device iZapper that captures a viewer's log in speci c contexts (e.g., I am at home, working, in the evening) and a viewer/recommender service iTube, which uses the input from the BeanCounter and iZapper in order to generate and present the recommended relevant television programmes to the user. iTube is typically a media centre environment, which can make use of the semantic recommendation service's output via an API. It could also be a display on a computer or a smaller device. Currently, we plan to use the iFanzy[2] recommendation service, although the architecture will allow for many di erent recommendations services to be used. 4 4.1
iZapper: the controlling device Traditionally televisions are controlled with dedicated remote controls. To develop a viewer's pro le, the viewer has to explicitly con gure it. iZapper (see Figure 4) is a native iPhone application intended to control a media centre (typical device and channel control), capture a viewer's context, and to record viewer's activities to gain more information. Several types of contextual information can be retrieved from a user's iPhone, such as location (via GPS or wi endpoint), period (via time) and optionally tasks (via agenda).
Television activities, such as watching, recording, ranking and bookmarking will be recorded. All this information will be pushed to the BeanCounter. Every viewer has access to his personal iZapper. The BeanCounter will use the iZapper as one source of user information for context, activities and pro le. The context is the circumstances and surroundings of a viewer, composed of location, period and task, and is relevant for the fact that viewers' ratings are potentially made to hold in a speci c context. 4.2
The BeanCounter itself can be modelled as a container of components (called tubelets) responsible for calling and managing a service the user wants to import into the system. Thus, each Web source (e.g., FriendFeed, Twitter, Glue, last.fm) is wrapped and accessed by a tubelet, that is speci cally built to handle data from that source. Whatever the format is adopted by the Web service response (e.g., ATOM, JSON or some sort of XML), a tubelet can parse such content and map it to RDF, allowing a native integration. All the logic for tubelet management are embedded in the container determining the life cycle of each tubelet.
The activation of each tubelet involves (
Another ow is more model-driven rather than source-driven. A container is devised, where a modelet is the equivalent of a tubelet. A modelet can accept structured user data in a cross-source fashion. Every time a new piece of information is needed, a new modelet is plugged into the modelets container. A modelet feeds then into the proper java beans, which just as for the rst ow are serialised and passed to a pipeline. A key aspect of the BeanCounter is illustrated by this last component. It allows for aggregation of data from di erent Web sources, where the same entity - movie, city, person - within two sources should be unambiguously identi ed and mapped in both sources. Adding a new modelet to the modelets container is possible by calling speci c APIs de ning the desired modelet (e.g., how it is structured, which elds) and then hot-plugging it into the container. 5
During the requirements phase and implementation of two BeanCounters and the
iZapper, we have identi ed several challenges to realise e ective reuse of Social
Web user data in domain-speci c recommender system (for TV programmes).
Some challenges have already been addressed in the development of
BeanCounters. From this work result also some practical guidelines for semantic data
integration. We now give a brief overview of those points, starting with Lessons
Learned for Semantic Data Aggregators:
{ Pipelines. To ingest data into the system through pipelines, which are
dynamically editable (e.g., add new pipe elements in speci c points of the
chain). Some pipes could be responsible for collecting statistics on certain
triple data, while others could handle the building of speci c indexes.
{ Container Programming Pattern. This pattern manages the addition
and removal of components, as well as their life cycle, inter-communication
and activation. It provides an abstraction that hides the complexity of the
system, allowing developers to ignore low level functionalities. Some system
components are subjected to change depending on the evolution of the
services they are written for, so the system should support the addition and
the replacement of dynamic components without requiring a restart.
{ Push, Pull and Trigger. A exible system should gather data (
Challenges for User Data Reuse on the Social Web: { Presentation of Aggregated User Data
How to maintain user's motivation to add new data and achieve good
con uence of the content and the context?
What are non obtrusive ways of making the user aware of the privacy
implications of her actions?
How to make user aware of the current (
How to keep user in control of pro les, content and context data used? { Privacy
How to handle privacy at the architectural level? How to achieve user management without asking users, e.g., for services username/password, or to create username/password for the BeanCounter? How to help users understand the privacy issues when reusing and aggregating their data?
How to manage data reuse, e.g., by using creative commons licensing? { Context Capturing
What are useful contexts, e.g., temporal, spatial, task-, device-related? How to capture user's context (semi-)automatically, i.e., minimise the explicit user input, while maximising the background collection of user and context data, as well as deductions from this data? { Data Aggregation and Enrichment
How to achieve uni ed and simple access to dynamic, growing and distributed multimedia content of diverse formats? How to determine, which information from the activity stream is useful for determining user's interests and what enrichment is relevant, useful and accurate in di erent user contexts? What is a minimal amount of data to start up the personalisation process, preventing a `cold start' ? How to represent user data statistics, strength of user interest and user context in machine-readable format, e.g., RDF? How to exploit current standards in television content metadata available both for providers and consumers
Our current experience of aggregating user data from various Social Web applications is presented in this paper. The BeanCounter is as a user data aggregation component for the NoTube architecture. The usage scenarios motivate the relation between TV recommendation and Social Web. To aggregate heterogeneous data, we have incorporated alignment support in our design and prototype implementation. Finally, we presented the lessons learned from the prototype implementation regarding the di erent challenges that need to be tackled and further work to be done.
This work is supported by the EU IP Project NoTube, see http://www.notube.tv.