Linked Open Data-enabled Strategies for Top-N Recommendations Cataldo Musto Pierpaolo Basile Pasquale Lops Dept. of Computer Science Dept. of Computer Science Dept. of Computer Science Univ. of Bari Aldo Moro, Italy Univ. of Bari Aldo Moro, Italy Univ. of Bari Aldo Moro, Italy cataldo.musto@uniba.it pierpaolo.basile@uniba.it pasquale.lops@uniba.it Marco de Gemmis Giovanni Semeraro Dept. of Computer Science Dept. of Computer Science Univ. of Bari Aldo Moro, Italy Univ. of Bari Aldo Moro, Italy marco.degemmis@uniba.it giovanni.semeraro@uniba.it ABSTRACT radio programs, genes, proteins, drugs, online communities, The huge amount of interlinked information referring to dif- statistical data, and reviews in a single global data space, ferent domains, provided by the Linked Open Data (LOD) the Web of Data [2]. initiative, could be e↵ectively exploited by recommender sys- This information, interlinked with each other, forms a tems to deal with the cold-start and sparsity problems. global graph called Linked Open Data cloud, whose nucleus In this paper we investigate the contribution of several is represented by DBpedia2 . features extracted from the Linked Open Data cloud to the A fragment of the Linked Open Data cloud is depicted in accuracy of di↵erent recommendation algorithms. We focus Figure 1. on the top-N recommendation task in presence of binary user feedback and cold-start situations, that is, predicting ratings for users who have a few past ratings, and predicting ratings of items that have been rated by a few users. Results show the potential of Linked Open Data-enabled approaches to outperform existing state-of-the-art algorithms. Categories and Subject Descriptors H.3.3 [Information Systems]: Information Search and Re- trieval Keywords Content-based Recommender Systems; Top-N recommenda- tions; Implicit Feedback; Linked Open Data; DBpedia 1. INTRODUCTION Figure 1: Fragment of the Linked Open Data cloud Recently, novel and more accessible forms of information (as of September 2011). coming from di↵erent open knowledge sources represent a rapidly growing piece of the big data puzzle. Using open or pooled data from many sources, often com- Over the last years, more and more semantic data are pub- bined and linked with proprietary big data, can help develop lished following the Linked Data principles1 , by connecting insights difficult to uncover with internal data alone [4], and information referring to geographical locations, people, com- can be e↵ectively exploited by recommender systems to deal panies, book, scientific publications, films, music, TV and with classical problems of cold-start and sparsity. 1 On the other hand, the use of a huge amount of inter- http://www.w3.org/DesignIssues/LinkedData.html linked data poses new challenges to recommender systems researchers, who have to find e↵ective ways to integrate such knowledge into recommendation paradigms. This paper presents a preliminary investigation in which we propose and evaluate di↵erent ways of including several kinds of Linked Open Data features in di↵erent classes of recommendation algorithms. The evaluation is focused on Copyright 2014 for the individual papers by the paper’s authors. the top-N recommendations task in presence of binary user Copying permitted for private and academic purposes. This volume is feedback and cold-start situations. published and CBRecSys copyrighted 2014, October 6, by its editors. 2014, Silicon Valley, CA, USA. 2 Copyright CBRecSys 2014 2014,byOctober the author(s). 6, 2014, Silicon Valley, CA, USA. http://dbpedia.org 49 This paper extends our previous work carried out to par- leveraged various features sets created from DBpedia. Ad- ticipate to the Linked Open Data-enabled Recommender ditional Linked Open Data sources were explored, such as Systems challenge3 [1], by presenting results for new tested British Library Bibliography4 and DBTropes5 , even though algorithms, along with the various combinations of features. they did not provide meaningful features with respect to Results show the potential of Linked Open Data-enabled ap- those derived from DBpedia. The results of the individual proaches to outperform existing state-of-the-art algorithms. recommenders were combined using stacking regression and rank aggregation using Borda. 2. RELATED WORK Previous attempts to build recommender systems that ex- 3. METHODOLOGY ploit Linked Open Data are presented in [17], where a music Section 3.1 describes the set of di↵erent features extracted recommender system uses DBpedia to compute the Linked from the Linked Open Data cloud, while Section 3.2 presents Data Semantic Distance, which allows to provide recommen- di↵erent kinds of recommendation algorithms, i.e. those dations by computing the semantic distance for all artists based on vector space and probabilistic models, those based referenced in DBpedia. on the use of classifiers, and graph-based algorithms, which In that work, the semantics of the DBpedia relations is not are fed in di↵erent ways by the features extracted from the taken into account, di↵erently from the approach described Linked Open Data cloud. in [5], where properties extracted from DBpedia and Linked- MDB [12] are exploited to perform a semantic expansion of the 3.1 Features extracted from the Linked Open item descriptions, suitable for learning user profiles. Data cloud In [15], DBpedia is used to enrich the playlists extracted The use of Linked Open Data allows to bridge the gap from a Facebook profile with new related artists. Each between the need of background data and the challenge to artist in the original playlist is mapped to a DBpedia node, devise novel advanced recommendation strategies. and other similar artists are selected by taking into account There are two main approaches to extract Linked Open shared properties, such as the genre and the musical cate- Data features to represent items: gory of the artist. DBpedia is also used in [16] to capture the complex rela- 1. use of the Uniform Resource Identifier (URI) tionships between users, items and entities by extracting the paths that connect users to items, in order to compute rec- 2. use of entity linking algorithms. ommendations through a learning to rank algorithm called SPRank. SPRank is a hybrid recommendation algorithm The first approach directly extracts DBpedia properties for able to compute top-N item recommendations from implicit each item by using its Uniform Resource Identifier (URI). feedback, that e↵ectively incorporates ontological knowledge URIs are the standard way to identify real-world entities, coming from DBpedia (content-based part) with collabora- and allow to define an entry point to DBpedia. tive user preferences (collaborative part) in a graph-based However, DBpedia provides a huge set of properties for setting. Starting from the common graph-based represen- each item, hence a proper strategy to select the most valu- tation of the content and collaborative data models, all the able ones is necessary. We could manually identify and select paths connecting the user to an item are considered in or- a subset of domain-dependent properties, or we could take der to have a relevance score for that item. The more paths into account a subset of the most frequent ones. between a user and an item, the more that item is relevant Referring to the book domain, in which we performed the to that user. evaluation, we selected the 10 properties in Table 1, which The increasing interest in using Linked Open Data to cre- are both very frequent and representative of the specific do- ate a new breed of content-based recommender systems is main. witnessed by the success of the recent Linked Open Data- Starting from these properties, further resources could be enabled Recommender Systems challenge held at the Euro- recursively added. For example, starting from a book, we pean Semantic Web Conference (ESWC 2014). The contest could retrieve its author through the property consisted of 3 tasks, namely rating prediction in cold-start http://dbpedia.org/ontology/author situations, top-N recommendation from binary user feed- and then retrieve and link other resources by the same au- back, and diversity. Interestingly, top-N recommendation thor, or other genres of works by the same author. from binary user feedback was the task with the highest As an example, the resulting representation obtained for number of participants. The best performing approach was the book The Great and Secret Show is provided in Figure based on an ensemble of algorithms based on popularity, 2. The book is linked to its author (Clive Barker ), to the Vector Space Model, Random Forests, Logistic Regression, genre (Fantasy literature), and to the Wikipedia categories and PageRank, running on a diverse set of semantic features (British fantasy novels and 1980s fantasy novels). Further- [1]. The performance of the single methods were aggregated more, other books by Clive Barker are reported, such as using the Borda count aggregation strategy. Most of the Books of Blood and Mister B. Gone. techniques used in the contest are presented in this paper. The second approach to extract LOD features uses entity Similarly to the best performing approach, the second best linking algorithms to identify a set of Wikipedia concepts performing one was based on the same ingredients [18]. In- occurring in the item description. Next, those Wikipedia deed, it combined di↵erent base recommenders, such as col- concepts can be easily mapped to the corresponding DBpedia laborative and content-based ones, with a non-personalized nodes. recommender based on popularity. Content-based strategies 4 http://bnb.data.bl.uk/ 3 5 challenges.2014.eswc-conferences.org/index.php/RecSys http://skipforward.opendfki.de/wiki/DBTropes 49 Property Description Frequency http://dbpedia.org/ontology/wikiPageWikiLink Link from a Wikipedia page to another Wikipedia 523,321 page. This property allows to take into account other Wikipedia pages which are somehow related. http://purl.org/dc/terms/subject The topic of a book. 38,627 http://dbpedia.org/property/genre The genre of a book. 12,488 http://dbpedia.org/property/publisher The publisher of a book. 9,798 http://dbpedia.org/ontology/author The author of a book. 8,669 http://dbpedia.org/property/followedBy The book followed by a specific book. 6,351 http://dbpedia.org/property/precededBy The book preceded by a specific book. 5,978 http://dbpedia.org/property/series The series of a book. 3,196 http://dbpedia.org/property/dewey The Dewey Decimal library Classification. 1,930 http://dbpedia.org/ontology/nonFictionSubject The subject of a non-fiction book 966 (e.g.: history, biography, cookbook, ...). Table 1: DBpedia properties selected for the book domain. Books_of _Blood author Clive_ Mister_ Barker author B._Gone author Figure 3: Tagme representation of the book The Great and Secret Show by Clive Barker. The_ Great_ genre Fantasy_ and_ literature All these features are used in di↵erent ways by the di↵er- Secret_ ent recommendation algorithms presented in the following show section. Details are reported in Section 4.2. subject subject 3.2 Recommendation Algorithms precededBy We tested three di↵erent classes of algorithms for generat- ing top-N recommendations, by using several combinations British_ 1980s_ fantasy_ fantasy_ of features extracted from the Linked Open Data cloud. novels novels Everville 3.2.1 Algorithms based on the Vector Space and Prob- abilistic Models Most content-based recommender systems rely on simple retrieval models to produce recommendations, such as key- Figure 2: Properties of the book The Great and Se- word matching or Vector Space Model (VSM). cret Show by Clive Barker. VSM emerged as one of the most e↵ective approaches in the area of Information Retrieval, thanks to its good com- promise between e↵ectiveness and simplicity. Documents and queries are represented by vectors in an n-dimensional Several techniques can be adopted, such as Explicit Se- vector space, where n is the number of index terms (words, mantic Analysis [8] or Tagme [7]. stems, concepts, etc.). In this work we adopt Tagme, that implements an anchor Formally, each document is represented by a vector of disambiguation algorithm to produce a Wikipedia-based rep- weights, where weights indicate the degree of association resentation of text fragments, where the most relevant con- between the document and index terms. cepts occurring in the text are mapped to the Wikipedia Given this representation, documents are ranked by com- articles they refer to. Tagme performs a sort of feature se- puting the distance between their vector representations and lection by filtering out the noise in text fragments, and its the query vector. Let D = {d1 , d2 , ..., dN } denote a set of main advantage is the ability to annotate very short texts. documents or corpus, and T = {t1 , t2 , ..., tn } be the dictio- As an example, the resulting representation obtained for nary, that is to say the set of words in the corpus. T is the book The Great and Secret Show is provided in Figure obtained by applying some standard natural language pro- 3. Interestingly, the technique is able to associate several cessing operations, such as tokenization, stopwords removal, concepts which are somehow related to the book, and which and stemming. Each document dj is represented as a vector could be useful to provide accurate and diverse recommen- in a n-dimensional vector space, so dj = {w1j , w2j , ..., dnj }, dations, as well. where wkj is the weight for term tk in document dj . The 50 most common weighting scheme is the TF-IDF (Term Frequency- Several works generally rely on the Rocchio algorithm [21] Inverse Document Frequency). to incrementally refine the user profiles by exploiting positive In content-based recommender systems relying on VSM, and negative feedback provided by users, even though the the query is the user profile, obtained as a combination of method needs an extensive tuning of parameters for being the index terms occurring in the items liked by that user, e↵ective. and recommendations are computed by applying a vector Negative relevance feedback is also discussed in [6], in similarity measure, such as the cosine coefficient, between which the idea of representing negation by subtracting an the user profile and the items to be recommended in the unwanted vector from a query emerged, even if nothing same vector space. about how much to subtract is stated. Hence, vector nega- However, VSM is not able to manage either the latent se- tion is built on the idea of subtracting exactly the right mantics of each document or the position of the terms occur- amount to make the unwanted vector irrelevant to the re- ring in it. Hence, we proposed an approach able to produce sults we obtain. a lightweight and implicit semantic representation of docu- This removal operation is called vector negation, which is ments (items and user profiles). The technique is based on related to the concept of orthogonality, and it is proposed in the distributional hypothesis, according to which “words that [23]. occur in the same contexts tend to have similar meanings” [11]. This means that the meaning of a word is inferred by 3.2.2 Algorithms based on Classifiers analyzing its usage in large corpora of textual documents, The recommendation process can be seen as a binary clas- hence words are semantically similar to the extent that they sification task, in which each item has to be classified as share contexts. interesting or not with respect to the user preferences. The gist of the technique is presented in [14], in which a We learned classifiers using two algorithms, namely Ran- novel content-based recommendation framework, called en- dom Forests (RF) [3] and Logistic Regression (LR). hanced Vector Space Model (eVSM), is described. eVSM RF is an ensemble learning method, combining di↵erent adopts a latent semantic representation of items in terms tree predictors built using di↵erent samples of the training of contexts, i.e. a term-context matrix is adopted, instead data and random subsets of the data features. The class of of the classical term-document matrix adopted in the VSM. an item is determined by the majority voting of the classes The advantage is that the context can be adapted to the spe- returned by the individual trees. The use of di↵erent sam- cific granularity level of the representation required by the ples of the data from the same distribution and of di↵erent application: for example, given a word, its context could sets of features for learning the individual trees prevent the be either a single word it co-occurs with, a sentence, or the overfitting. whole document. LR is a supervised learning method for classification which The use of fine-grained representations of contexts calls builds a linear model based on a transformed target variable. for specific techniques for reducing the dimensionality of vec- tors. Besides the classical Latent Semantic Indexing, which 3.2.3 Graph-based Algorithms su↵ers of scalability issues, more scalable techniques were We adopted PageRank with Priors, widely used to obtain investigated, such as Random Indexing [22], adopted in the an authority score for a node based on the network con- eVSM model. nectivity. Di↵erently from PageRank, it is biased towards Random Indexing in an incremental method which allows the preferences of a specific user, by adopting a non-uniform to reduce a vector space by projecting the points into a ran- personalization vector to assign di↵erent weights to di↵erent domly selected subspace of enough high dimensionality. The nodes [13]. goal of using eVSM is to compare a vector space represen- In order to run the PageRank, we need to represent data tation which adopts very few dimensions for representing using a graph model. To this purpose, users and items in items, with respect to a classical VSM. the dataset are represented as nodes of a graph, while links As an alternative to VSM, we used the BM25 probabilistic are represented by the positive users’ feedback. The graph model [19], one of the most dominant retrieval paradigm may be enriched in di↵erent ways, for example exploiting today. The ranking function for matching a query q (user entities and relations coming from DBpedia: in this case the profile) and an item I is: whole graph would contain nodes representing users, items, and entities, and edges representing items relevant to users, X nt · (↵ + 1) and relations between entities. This unified representation R= |I| · idf (t) (1) allows to take into account both collaborative and content- t2q nt + ↵ · (1 + avgdl ) based features to produce recommendations. nt is frequency of t in the item I, ↵ and are free parame- In the classic PageRank, the prior probability assigned to ters, avgdl is the average item length, and idf (t) is the IDF each node is evenly distributed ( N1 , where N is the number of of feature t: nodes), while PageRank with Priors is biased towards some nodes, i.e. the preferences of a specific user (see Section 4.2). N df (t) + 0.5 idf (t) = log (2) df (t) + 0.5 4. EXPERIMENTAL EVALUATION df (t) is the number of items in which the feature t occurs, The goal of the experiments is to evaluate the contribu- N is the cardinality of the collection. tion of diverse combinations of features, including those ex- For all the previous models we explicitly managed neg- tracted from the Linked Open Data cloud, to the accuracy ative preferences of users by adopting the vector negation of di↵erent classes of recommendation algorithms. operator proposed in [23], based on the concept of orthogo- The experiments that have been carried out try to answer nality between vectors. to the following questions: 51 1. Which is the contribution of the Linked Open Data training examples, while these did not provide valuable re- features to the accuracy of top-N recommendations sults for RF. algorithms, in presence of binary user feedback and The RF classifier used 1,500 trees to provide a good trade- cold-start situations? o↵ between accuracy and efficiency. For Logistic Regression we adopted the implementation 2. Do the Linked Open Data-enabled approaches out- provided by Liblinear6 , while for Random Forests we adopted perform existing state-of-the-art recommendation al- the implementation provided by the Weka library7 . gorithms? Top-N recommendations are produced by ranking items according to the probability of the class. 4.1 Dataset The dataset used in the experiment is DBbook, coming Graph-based Algorithms. from the recent Linked-Open Data-enabled Recommender PageRank with Priors is performed (for each single user) Systems challenge. It contains user preferences retrieved using graphs with di↵erent sets of nodes. Initially, only from the Web in the book domain. Each book is mapped users, items and links represented by the positive feedback to the corresponding DBpedia URI, which can be used to are included; next, we enriched the graph with the 10 prop- extract features from di↵erent datasets in the Linked Open erties extracted from DBpedia (see Section 3.1). Then, we Data cloud. ran a second level expansion stage of the graph to retrieve The training set released for the top-N recommendation the following additional resources: task contains 72,372 binary ratings provided by 6,181 users on 6,733 items. The dataset sparsity is 99.83%, and the 1. internal wiki links of the new added nodes distribution of ratings is reported in Table 2. The test set contains user-item pairs to rank in order to 2. more generic categories according to the hierarchy in produce a top-5 item recommendation list for each user, to DBpedia be evaluated using F1@5 accuracy measure. 3. resources of the same category 4.2 Experimental setup 4. resources of the same genre Each recommendation algorithm is fed by a diverse set of 5. genres pertaining to the author of the book features. Besides TAGME and LOD features, algorithms may also use 6. resources written by the author BASIC features, i.e. number of positive, number of nega- tive, and total number of feedbacks provided by users and 7. genres of the series the book belongs to. provided on items, ratio between positive, negative and to- tal number of feedbacks provided by users and provided on This process adds thousands of nodes to the original graph. items and CONTENT features, obtained by processing book de- For this reason, we pruned the graph by removing nodes scriptions gathered from Wikipedia. A simple NLP pipeline which are neither users nor books and having a total num- removes stopwords, and applies stemming. For books not ber of inlinks and outlinks less than 5. This graph eventually existing in Wikipedia, DBpedia abstracts were processed. consisted of 340,000 nodes and 6 millions links. For all the methods, the 5 most popular items are assigned The prior probabilities assigned to nodes depend on the as liked to users with no positive ratings in the training set. users’ preferences, and are assigned according to the follow- Indeed, 5.37 is the average number of positive ratings for ing heuristics: 80% of the total weight is evenly distributed each user in the dataset (see Table 2). among items liked by users (0 assigned to disliked items), 20% is evenly distributed among the remaining nodes. We Algorithms based on the Vector Space and Probabilis- ran the algorithm with a damping factor set to 0.85. tic Models. We adopted the implementation of PageRank provided by Recommender systems relying on VSM and probabilistic the Jung library8 . framework index items using CONTENT, TAGME and LOD fea- The PageRank computed for each node is used to rank tures, and use as query the user profile obtained by combin- items in the test set. ing all the index terms occurring in the items liked by that user. 4.3 Results Items in the test set are ranked by computing the simi- Figure 4 shows the results of VSM and probabilistic mod- larity with the user profile. For VSM and eVSM the cosine els. The paired t-test is used for testing the significance. measure is adopted, while Equation 1 is used for the proba- The first interesting outcome is that the worst configu- bilistic model. According to the literature [20], parameters rations are always obtained using LOD data alone, and the ↵ and are set to 1.6 and 0.75, respectively. best ones always contain TAGME features. In detail, the best VSM configuration is obtained combining TAGME and LOD features, which is significantly better than using LOD fea- Algorithms based on Classifiers. tures alone (p < 0.0001) and CONTENT alone (p < 0.05). The Classifiers based on Random Forests and Logistic Regres- combination of CONTENT and LOD features outperforms the sion are trained with examples represented using CONTENT, models using just CONTENT (p < 0.01) or just LOD features TAGME and LOD features, and labeled with the binary ratings (p < 0.001). provided by users. The value of each feature is the num- 6 ber of times it occurs in each item, normalized in the [0,1] www.csie.ntu.edu.tw/˜cjlin/liblinear/ 7 interval. www.cs.waikato.ac.nz/ml/weka/ 8 The LR classifier always includes BASIC features in the jung.sourceforge.net 52 Property Value Statistics about users Avg. ratings provided by users 11.71 (5.37 positive, 6.34 negative) # of users who provided only negative ratings 520 (8.41%) # of users having a number of positive ratings below the avg. 3,804 (61.54%) # of users having more negative than positive ratings 3,343 (54.09%) Statistics about items Avg. ratings received by items 10.75 (4.93 positive, 5.82 negative) # of items with no positive ratings 1,839 (27.31%) # of items having a number of positive ratings below the avg. 6,447 (95.75%) # of items having more negative than positive ratings 4,046 (60.09%) Table 2: Distribution of ratings for the DBbook dataset. VectorSpaceandProbabilisticModels Classifiers 0,5600 0,5600 0,5500 0,5500 0,5400 0,5400 F1 F1 0,5300 0,5300 VSM 0,5200 0,5200 LR eVSM 0,5100 0,5100 RF BM25 0,5000 0,5000 Figure 4: Results of the VSM and probabilistic mod- Figure 5: Results of the classifiers using di↵erent els using di↵erent combinations of features. combinations of features. worse than vector space and probabilistic models. The best The best configuration for eVSM adopts TAGME features result is obtained using ALL features. Since Random Forests alone, and is significantly better than all the configurations classifiers are able to automatically perform feature selec- but the one combining CONTENT and TAGME features (p = tion, this was an unexpected result which deserves further 0.13). This could mean that the entity linking algorithm investigations. is able to select the most important features in the book Finally, Figure 6 presents the results obtained by the PageR- descriptions, while CONTENT features introduce noise. ank with Priors algorithm. For BM25, the best configuration with ALL the features significantly outperforms all the others but the one combin- ing CONTENT and LOD features (p = 0.53). PageRankwithPriors Surprisingly, there is no statistical di↵erence between the 0,5600 140 best performing configuration for VSM and the best one for 0,5500 120 BM25. 0,5400 100 Hours A final remark is that eVSM performance is not compara- 80 F1 0,5300 ble to the other methods, even though it is worth noting that 0,5200 60 40 it represents items using very low-dimensional vectors (di- 0,5100 20 mension=500), compared to VSM, which uses vectors whose 0,5000 0 dimensionality is equal to the number of items (6,733). NoContent 10properties 10properties+ 10properties+ expansionstage+ expansionstage Figure 5 presents the results obtained by the classifiers. pruning We note that Logistic Regression always outperforms Ran- PageRank Executiontime dom Forests, and provides better results than the vector space and probabilistic models, regardless the set of adopted features. The best result using Logistic Regression is obtained with Figure 6: Results of the PageRank with Priors using TAGME features alone. This configuration significantly out- di↵erent combinations of features. performs the one including CONTENT and LOD features (p < 0.05), while it is not di↵erent with respect to the other con- When using PageRank with Priors, we observe the impact figurations. This is probably due to the high sparsity of of the graph size on both the accuracy and execution time. the feature vector used to represent each training example Starting with a graph not including content information, we (220,000 features). observe the worst performance and the lowest execution time Random Forests classifiers outperform eVSM, but they are (2 hours on an Intel i7 3Ghz 32Gb RAM - the algorithm 53 is performed for each user with di↵erent weights initially However, in order to generalize our preliminary results, it assigned to the nodes). is necessary to further investigate: Enriching the graph with the 10 selected DBpedia proper- ties leads to an improvement of accuracy (p < 0.001), and to • the e↵ect of di↵erent levels of sparsity on the recom- a 5 hours execution time. Running the expansion stage and mendation accuracy: to this purpose, it is needed to pruning of nodes as described in Section 4.2, the time needed assess the extent to which LOD features are able to im- to run the algorithm increases to 14 hours and produces a prove the performance of recommendation algorithms slight accuracy improvement (p < 0.001). Results using the for di↵erent levels of sparsity graph with no pruning procedure are not di↵erent from the • the accuracy on other datasets to generalize our con- previous method (p = 0.09), but its time complexity is not clusions: further experiments on di↵erent target do- acceptable. This call for a more efficient implementation of mains are needed. Indeed, di↵erent item types, such the algorithm. as books, movies, news, songs have di↵erent character- To complete the empirical evaluation, we compare the best istics which could lead to di↵erent results. Moreover, performing configuration of each algorithm in each class, experiments on a much larger scale are needed with some state-of-the-art algorithms. More specifically, we report the performance of user-to- • the e↵ect of the selection of domain-specific DBpedia user and item-to-item collaborative filtering, besides two non- properties to feed the recommendation algorithms: it personalized baselines based on popularity and random rec- is needed to assess the e↵ect of the selection of spe- ommendations. cific sets of properties on the performance of the rec- Furthermore, we report the results for two algorithms for ommendation algorithms. Indeed, DBpedia contains a top-N recommendations from implicit feedback: an exten- huge number of properties, and their selection could sion of matrix factorization optimized for Bayesian Personal- have a strong influence on the accuracy of the rec- ized Ranking (BPRMF ) [9] and SPRank [16], able to exploit ommendation methods. Our preliminary experiments LInked Open Data knowledge bases to compute accurate leverage 10 DBpedia properties which are both frequent recommendations. and representative of the specific domain, but a subset Except for SPRank, we used the implementations avail- of these properties, or a di↵erent set of features could able in MyMediaLite 3.10 [10], using the default parameters. lead to di↵erent results. The analysis of results in Figure 7 unveils the difficulty of collaborative filtering algorithms to deal with the high As future work, we will study the e↵ect of enriching the sparsity of the dataset (99.83%), and with the high number graph-based representation with DBpedia nodes extracted of users who provided only negative preferences, or more from the Tagme entity linking algorithm. negative than positive ratings. It is unexpected the bet- Indeed, using entity linking to access DBpedia knowledge ter performance of BPRMF compared to SPRank, di↵er- is innovative and avoids the need of explicitly finding URIs ently from previous results obtained on the MovieLens and for items, a complex process which may hinder the use of Last.fm datasets [16]. It is also surprising the better perfor- the Linked Open Data. Hence, the use of entity linking algo- mance of simple algorithms based on the vector space and rithms represents a novel way to access the DBpedia knowl- probabilistic models with respect to matrix factorization. edge through the analysis of the item descriptions, without exploiting any explicit mapping of items to URIs. Furthermore, starting from the preliminary evaluation car- OverallComparison ried out in [1], we will thoroughly investigate the potential 0,5600 of using the wealth of relations of LOD features to produce not only accurate, but also diversified recommendation lists. 0,5500 0,5400 Acknowledgments This work fulfils the research objectives of the project “VIN- F1 0,5300 0,5200 CENTE - A Virtual collective INtelligenCe ENvironment 0,5100 to develop sustainable Technology Entrepreneurship ecosys- tems” (PON 02 00563 3470993) funded by the Italian Min- 0,5000 istry of University and Research (MIUR). 5. REFERENCES [1] P. Basile, C. Musto, M. de Gemmis, P. Lops, F. Narducci, and G. Semeraro. Aggregation strategies Figure 7: Comparison with other state-of-the-art for linked open data-enabled recommender systems. In approaches. European Semantic Web Conference (satellite Events), 2014. [2] C. Bizer. The emerging web of linked data. IEEE 4.4 Discussion Intelligent Systems, 24(5):87–92, 2009. The analysis of the previous results allows to conclude [3] L. Breiman. Random forests. Machine Learning, that TAGME and LOD features have the potential to improve 45(1):5–32, 2001. the performance of several recommendation algorithms for [4] M. Chui, J. Manyika, and S. V. Kuiken. What computing top-N recommendations from binary user feed- executives should know about open data. McKinsey back. Quarterly, January 2014, 2014. 54 [5] T. Di Noia, R. Mirizzi, V. C. Ostuni, and D. Romito. data. In European Semantic Web Conference (Satellite Exploiting the web of data in model-based Events), 2014. recommender systems. In P. Cunningham, N. J. [19] S. E. Robertson, S. Walker, M. H. Beaulieu, A. Gull, Hurley, I. Guy, and S. S. Anand, editors, Proceedings and M. Lau. Okapi at TREC. In Text REtrieval of the ACM Conference on Recommender Systems ’12, Conference, pages 21–30, 1992. pages 253–256. ACM, 2012. [20] S. E. Robertson and H. Zaragoza. The probabilistic [6] M. D. Dunlop. The e↵ect of accessing nonmatching relevance framework: Bm25 and beyond. Foundations documents on relevance feedback. ACM Trans. Inf. and Trends in Information Retrieval, 3(4):333–389, Syst., 15:137–153, 1997. 2009. [7] P. Ferragina and U. Scaiella. Fast and Accurate [21] J. Rocchio. Relevance Feedback Information Retrieval. Annotation of Short Texts with Wikipedia Pages. In Gerald Salton, editor, The SMART retrieval system IEEE Software, 29(1):70–75, 2012. - experiments in automated document processing, pages [8] E. Gabrilovich and S. Markovitch. Wikipedia-based 313–323. Prentice-Hall, Englewood Cli↵s, NJ, 1971. semantic interpretation for natural language [22] M. Sahlgren. An introduction to random indexing. In processing. Journal of Artificial Intelligence Research Proc. of the Methods and Applications of Semantic (JAIR), 34:443–498, 2009. Indexing Workshop at the 7th Int. Conf. on [9] Z. Gantner, L. Drumond, C. Freudenthaler, S. Rendle, Terminology and Knowledge Engineering, 2005. and L. Schmidt-Thieme. Learning attribute-to-feature [23] D. Widdows. Orthogonal negation in vector spaces for mappings for cold-start recommendations. In G. I. modelling word-meanings and document retrieval. In Webb, B. Liu, C. Zhang, D. Gunopulos, and X. Wu, Proceedings of the 41st Annual Meeting of the editors, 10th IEEE International Conference on Data Association for Computational Linguistics, pages Mining, pages 176–185. IEEE Computer Society, 2010. 136–143, 2003. [10] Z. Gantner, S. Rendle, C. Freudenthaler, and L. Schmidt-Thieme. Mymedialite: a free recommender system library. In B. Mobasher, R. D. Burke, D. Jannach, and G. Adomavicius, editors, Proceedings of the ACM Conference on Recommender Systems ’11, pages 305–308. ACM, 2011. [11] Z. S. Harris. Mathematical Structures of Language. Interscience, New York 1968. ” [12] O. Hassanzadeh and M. P. Consens. Linked movie data base. In C.Bizer, T. Heath, T. Berners-Lee, and K. Idehen, editors, Proceedings of the WWW2009 Workshop on Linked Data on the Web, LDOW 2009, volume 538 of CEUR Workshop Proceedings. CEUR-WS.org, 2009. [13] T. H. Haveliwala. Topic-Sensitive PageRank: A Context-Sensitive Ranking Algorithm for Web Search. IEEE Trans. Knowl. Data Eng., 15(4):784–796, 2003. [14] C. Musto. Enhanced vector space models for content-based recommender systems. In Proceedings of the ACM Conference on Recommneder Systems ’10, pages 361–364. ACM, 2010. [15] C. Musto, G. Semeraro, P. Lops, M. de Gemmis, and F. Narducci. Leveraging social media sources to generate personalized music playlists. In C. Huemer and P. Lops, editors, E-Commerce and Web Technologies - 13th International Conference, EC-Web 2012, volume 123 of Lecture Notes in Business Information Processing, pages 112–123. Springer, 2012. [16] V. C. Ostuni, T. Di Noia, E. Di Sciascio, and R. Mirizzi. Top-n recommendations from implicit feedback leveraging linked open data. In Q. Yang, I. King, Q. Li, P. Pu, and G. Karypis, editors, Proceedings of the ACM Conference on Recommender Systems ’13, pages 85–92. ACM, 2013. [17] A. Passant. dbrec - Music Recommendations Using DBpedia. In International Semantic Web Conference, Revised Papers, volume 6497 of LNCS, pages 209–224. Springer, 2010. [18] P. Ristoski, E. L. Mencia, and H. Paulheim. A hybrid multi-strategy recommender system using linked open 55