In this paper we present yourHistory: a Facebook application that aims to generate a tailor-made, personalized timeline of historic events, by matching a semantically enriched Facebook pro le to a pool of candidate historic events extracted from DBPedia. Two aspects are central to our application: (i) semantic linking technologies backed by rich open web knowledge bases for generating semantically enriched user pro les, and (ii) semantic relatedness metrics for ranking historic events to user pro les. This paper describes the development of a Facebook application that aims to be engaging for users, whilst at the same time being a source for data that can be applied to evaluating or improving the application. We describe our Wikipedia-based semantic relatedness metric for event ranking, but also the restrictions and constraints concerning privacy-sensitive and ethical matters, around data storage and user consent. Finally, we re ect on how this type of user data can be applied for evaluating or improving both the semantic linking and event ranking methods in future work.
H.4 [Information Systems Applications]: Miscellaneous
Today in history education, students are encouraged to study relations and coherence between events, discern patterns from global history, understand context and `see the bigger picture'. Looking at historical timelines is interesting in this regards, because it can relate historic events to one another and provide a sense of overview and context. However, with timelines of history students are still looking at the events from a distance, whilst teachers now often teach history starting from what students know and what they are interested in. By connecting history with the lives and interests of students teachers aim to make history more tangible, attractive and accessible.
yourHistory is a Facebook application (available at http: //apps.facebook.com/yourhistory) that aims to serve a tailor-made, personalized timeline of historic events, by leveraging a Facebook user's interests and pro le. The yourHistory timeline displays historical events1 side-by-side to historic events that are deemed relevant or interesting to the user. These can be smaller scale events that typically escape history books. By embedding the historic events that match a user's pro le in the wider context of the history of the 20th century, yourHistory encourages students to explore, relate events to each other, and put them into context of time periods and their personal interests. Whether in the classroom or at home, exploring historic events and putting them into context of their own life, allows anyone who is interested in history to discover new connections and links between events, time periods and people. To generate this tailor-made timeline, yourHistory leverages rich structured data from online, openly accessible knowledge bases.
The interaction of the user with the application can provide valuable signals on the inner workings of the application. By inviting the user to explore and interact with the yourHistory timeline, and storing these interactions (in the form of clicks), we automatically aggregate data that can be later used for either evaluating or improving the application.
The rest of the paper is structured as follows. In Section 3 we describe our technical approach which involves semantic linking of Facebook user pro les, and retrieving candidate historic events from DBPedia (Section 3.1). Next, we describe our method of ranking candidate events to user proles in Section 3.2. In Section 4 we discuss some of the technical, privacy-related and ethical challenges we faced during the development of a live Facebook application. Finally, we brie y re ect on the possibilities and uses of logging user interactions with a Facebook application like yourHistory, for evaluation and online learning purposes, in Section 5. 2.
Central to the yourHistory application are two aspects: semantic linking for enriching user pro les, and semantic relatedness metrics to leverage the enriched pro les for event ranking. 2.1
Semantic linking is the task of identifying and linking mentions of concepts in raw text, to their referent concepts that are described in a Knowledge Base (KB). As in so-called
of_modern_history
Wiki cation [
Semantic linking has recently seen a surge in interest; it is
a focal point in evaluation campaigns such as the Text
Analysis Conference Knowledge Base Population (KBP) track.2
Consequently, it has seen a wide array of applications, from
enriching microblog posts [
Event retrieval is the task of retrieving (pages describing) events from a KB in response to an explicit query or an implicit one (such as a user's Facebook pro le). We consider the event retrieval task as a ranking problem, where our aim is to rank events on descending order of \relatedness" to the user. This is in contrast to, e.g., approaches of collaborative ltering, where the ties in a social network is the main focus for recommending items.
Central to our method of matching user pro les to candidate events is the notion of semantic relatedness between (Wikipedia) concepts, or in the case of yourHistory, the relatedness between candidate historic events and user proles. The intuition and our underlying assumption is: the more related an event is to a pro le entity, the more interesting it is to the user.
To compute this semantic relatedness between events and
user pro les, we combine methods that leverage Wikipedia's
structure with textual similarity approaches, and aggregate
for each event the semantic relatedness scores to all user
prole entities. There is a rich history of leveraging Wikipedia
to compute semantic similarity; an example is ESA
(Explicit Semantic Analysis) [
In order to generate a personalized timeline of historic events we need to consider how we de ne this personalization, i.e. how to identify events that are interesting or rele
vant to a particular user given its pro le.
We describe our application in four parts, rst our preprocessing approach (Section 3.1), next our Wikipedia-based semantic similarity event ranking method (Section 3.2), then we present yourHistory's interface (Section 3.3), and nally, we describe some of the practical implementation details concerning data and infrastructure (Section 3.4). 3.1
In this section we describe the preprocessing procedure of: (i) generating bag-of-concepts user pro les by applying semantic linking (Section 3.1.1), and (ii) extracting a list of candidate historic events from DBPedia (Section 3.1.2). 3.1.1
Once the user has given consent for obtaining data from their Facebook pro le (described in Section 4), yourHistory receives the user's pro le information in JSON-format through the Facebook API. We extract the values of several elds of the user's likes, movies, music, tv shows, bio information, and work and education history, to yield an initial bag-ofwords user pro le. The resulting bag-of-words user pro le contains all useful textual data from the users pro le and is then linked to the referent Wikipedia concepts using the semanticizer.3 The resulting semantically enriched bag-ofconcepts user pro le forms the basis for yourHistory's event matching process.
To minimize noise (i.e., wrongfully linked concepts) we
consider our entity linking framework's con dence score, by
setting a threshold on the SENSEPROB weight [
By representing a user pro le as a bag-of-concepts pro le, arguably we lose potentially valuable signals that could aid in the event ranking. An example is the temporal dimension; knowing where the user lived or worked at which point in time could prove useful. However, depending on the user's Facebook pro le, sparsity issues (i.e. few likes) withheld us from exploring more ne-grained or detailed pro ling approaches. 3.1.2
DBPedia is a structured representation of Wikipedia [
classes/ issuing queries to the public DBPedia SPARQL endpoint5 for concepts that belong to the dbpedia-owl:Event class, concepts that have a startDate or xsd:date-property with a value between 1900-01-01 and today. At the time of research/writing, we ended up with a total of 10,272 candidate events after issuing these queries. To enable us to match events to the bag-of-concepts user pro les, we map the retrieved DBPedia events to their Wikipedia equivalents. Finally, we have a pool of candidate events (E) represented by Wikipedia concepts, and a bag-of-concepts user pro le (U ), similarly consisting of Wikipedia concepts. This allows us to easily compare both. See Table 1 for an overview of the terminology used in this paper. 3.2
Here we describe how we rank events to user pro les, using our Wikipedia-based semantic relatedness-score S. The relatedness score S is calculated for each cuser in U to each cevent in E and consists of several signals. When these scores are computed for all possible (cuser; cevent)-pairs, we sum the resulting values and min-max normalize them, to yield for each cevent a score between 0 and 1, which represents its \semantic relatedness" to U . The relatedness is based on the following signals: 1. Whether or not the pro le concept and event are directly linked: 1 when either one's Wikipedia page con
tains a link to the other, 0 otherwise. 2. The link overlap between cuser and cevent: how many linked pages do both concepts share. 3. The textual similarity between the abstracts of the corresponding Wikipedia pages of cuser and cevent. For each cevent 2 E we sum the scores stemming from the di erent signals, to each cuser 2 U , and yield the nal cevent relatedness-score. These signals are further detailed below. 3.2.1
The rst signal is a binary value, representing whether or not the pro le concept cuser occurs in the set of outlinks of the event cevent, or vice versa. The intuition is that events that are directly linked to pro le concepts are more (directly) related to the user's pro le. 3.2.2
For the second signal we extract the set of outlinks of both cuser and cevent (i.e., all Wikipedia pages that are (hyper)linked in the concept's Wikipedia page). For performance reasons we create an index of virtual documents, generated by concatenating the IDs of the outlink set for each event. By then considering the (concatenated) set of pro le outlinks a query, we quickly retrieve the most similar events. 3.2.3
For our third and nal signal, we measure the cosine
similarity between the TF-IDF weighted vectors representing
the abstract of the concepts' Wikipedia pages. We use the
gensim topic modeling framework for this comparison [
The nal output is a ranked list of JSON objects representing events, containing the following properties: event date, event id, event title, event url, score, related entity id, related entity title, and related entity url. 3.3
yourHistory’s interface
Given the nal ranked list of cevent, we draw the events in a timeline using a D3.js timeline visualization. A screenshot of this timeline visualization is shown in Figure 1. We visually distinguish between the two types of events; shown in blue are the personalized historic events, and the central historical events are shown in gray. Clicking any of the events opens the corresponding Wikipedia page in a new window. The blue bar can be extended or reduced in size, to zoom in or out in time while maintaining a sense of context. Dragging the bar allows the user to move the frame through the timeline.
Currently, the events are shown as-is, but displaying some of the mechanics behind the event ranking (e.g., showing the user that her timeline contains event Y because she likes band X) might increase engagement and enable more valuable feedback. Including more information, and relaying it back to the user might increase engagement. 3.4
In this section we describe some of the practical implementation details: the datasets we use and its infrastructure. 3.4.1
yourHistory makes use of Wikipedia and its structured counterpart DBPedia. For semantic linking, yourHistory has access to a Wikipedia dump of March 4th, 2013. For candidate event retrieval, yourHistory queries the live DBPedia SPARQL endpoint.6 3.4.2
yourHistory consists of three components:
A back-end, powered by a Python Flask application A data repository: two MongoDB databases, one where we store user pro les, and another where users' interactions are stored A front-end, which consists of a web page containing an interactive timeline visualization, powered by D3.js. This web page is shown to the user inside the Facebook application.
The back-end handles communication with the Facebook API, data preprocessing, semantic linking and event scoring and ranking. It outputs a timeline of ranked events in JSON format to the D3.js JavaScript application that runs in the web interface, and draws the interactive timeline. We log the users' clicks by storing for each event clicked its unique identi er, and the time-stamp of the click.
When setting up a live Facebook application for running user studies, we are faced with speci c constraints and challenges, both technical ones and ethical, privacy-related ones. The need for real-time processing is an example of a technical constraint, while there are more privacy-related questions and issues related to data storage. These and other constraints are described in Section 4. This section consists of two parts: preliminary challenges faced concerning user's privacy and ethics when using Facebook for running online
user studies (Section 4.1), as well as the technical challenges and constraints faced when working on a live, real-time Facebook application (Section 4.2). 4.1
In the following we provide a brief guideline on how to ful ll the privacy requirements of user experiments. 4.1.1
Ethically, collecting data of minors is questionable. Facebook handles a minimum age restriction of 13. In the case of yourHistory, the ethical review board of the University of Amsterdam required the application to be restricted to users over 18 years of age. However, providing a di erent birthday is an easy way to circumvent this restriction, so we can not be guaranteed to rely on the birthday information from Facebook alone. We rely on the following heuristics as a further check: if the birthday extracted from the Facebook pro le identi es the user as below 18, the yourHistory application cannot be accessed. Next, the user is asked to declare she is over 18 years old in the yourHistory welcome screen, if the user here does not declare this, yourHistory will likewise not launch. 4.1.2
Facebook provides examples on how to use the Facebook API.7 Important aspects to note, privacy-wise, are the scope of the access of user data, as well as the transfer of this user data. The scope of access is set by the developer, and determines the type of user data the application can access, after the user grants the application permission to access their data. It includes such types as biography information, such as the user's birthday (user birthday), relationship status (relationship status), information regarding the user's religious and/or political beliefs (user religion politics), but also the user's list of friends (user friends), the posts on his wall (which include messages, photos and links shared), and the user's likes. Finally, Facebook enforces the data transfer over a secure connection (through an SSL certi ed server), assuring a safe transfer of data. 4.1.3
There are two stages in requesting user consent. The rst stage of user consent is initiated by Facebook when the user accesses the yourHistory application. In this stage, users are asked by Facebook if they agree with the scope of data access. We aim for a more informed consent where the user is more elaborately informed about the storage of the data as well as using the data for improving the application { currently, this is restricted to the second stage of requesting user consent.
Following guidelines provided by the ethical review board of the University of Amsterdam, we designed the next step of asking user consent. This next stage is initiated by the application itself, where we welcome the user with the dialog window shown in Figure 2. After a brief explanation of the app and its authors, the user is given a choice concerning data usage. The user may choose to have data accessed only by the authors of the application, by the authors and other researchers at the University of Amsterdam, or by nobody but the application. Concerning data storage (if the user
friend-smash/ has complied), we o er the user a choice between allowing the aggregated pro le to be stored for an inde nite amount of time, for 3 months at most, or for a single day. In any case, the data will never leave the University of Amsterdam. Finally, once the user declares she is over 18 years old, she is directed to yourHistory's main interface. 4.2
Due to the live and real-time nature of this application, an important constraint is that the application has to run fast and be responsive. The event ranking part in particular proved challenging in this regard: the numerous pairwise comparisons between user pro le concepts and candidate events are demanding. We addressed this primarily by optimizing the link overlap calculation procedure, the most compute-intensive operation, as described in Section 3.2.
However, numerous additional improvements towards speeding up the application could be considered, e.g., by downsizing the search space in which to make pair-wise comparisons, by clustering events and user pro le concepts before computing relatedness. The clustering could be focused on the content of the concepts (e.g., cluster categorically similar concepts), or a temporal dimension or range.
Since evaluation of the applications' performance is still work in progress, in this section we brie y re ect on the potential of using stored user interactions for this purpose. By storing the user's interactions with the timeline (in the form of clicks on events), we have access to a valuable signal of (implicit) feedback. This signal could be used for two goals: evaluating the application, and improving the application through (online) learning. We elaborate on both applications in the sections that follow. 5.1
An example application of analyzing user interactions is the evaluation of our semantic relatedness scoring function. In this case, we consider clicks on events as positive feedback. The intuition is that in yourHistory's goal of serving a personalized-timeline, inviting users to explore and learn, clicks represent the user's interest in an event, or can be considered an instantiation of the user's intent to read more: a measurable signal of user engagement.
By feeding this signal back to the scoring function, we can analyze whether the ranking correlates with clicks (i.e. do higher ranked events generate more interactions?), or whether the individual scoring functions' rankings might be more indicative (i.e. does scoring function #1 rank the more frequently clicked events higher than scoring function #2?). In the setting where we consider clicks positive feedback, and we aim to optimize the application for clicks to increase user engagement, we can additionally get insights into in what way combining various scoring functions is most e ective. 5.2
The current event ranking method is in a way \static": we rank event entities based on the individual relatedness to pro le concepts. Next to hand-tuning the algorithms based on clicks as described in the previous section, the user feedback might too be applied in an online learning setting, to automatically improve the application. Here the scoring functions will be no longer used to compute a de nitive score, but rather as features for a machine learning model. The positive feedback (clicks) can then be applied for training the model, enabling the application to learn to rank event entities given a user pro le. Additionally, the current feedback signal can be extended towards being explicit. A possible use-case for explicit user feedback is improvement of the semantic linking component. This component now too is static: we apply a threshold on the entity linker's con dence score. A possible extension would be to ask the user to judge their generated bag-of-concepts user pro le by removing or adding concepts that she does or does not feel associated with, and correcting wrongfully linked concepts. This explicit feedback can be used to improve the semantic linking, in an online learning setting similar to the one described above.
This research was partially supported by the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement nr 288024 (LiMoSINe project), the Netherlands Organisation for Scienti c Research (NWO) under project nrs 640.004.802, 727.011.005, 612.001.116, HOR11-10, the Center for Creation, Content and Technology (CCCT), the QuaMerdes project funded by the CLARINnl program, the TROVe project funded by the CLARIAH program, the Dutch national program COMMIT, the ESF Research Network Program ELIAS, the Elite Network Shifts project funded by the Royal Dutch Academy of Sciences (KNAW), the Netherlands eScience Center under project number 027.012.105 and the Yahoo! Faculty Research and Engagement Program.