The huge amount of interlinked information referring to different domains, provided by the Linked Open Data (LOD) initiative, could be e↵ ectively exploited by recommender systems to deal with the cold-start and sparsity problems. In this paper we investigate the contribution of several features extracted from the Linked Open Data cloud to the accuracy of di↵ erent recommendation algorithms. We focus on the top-N recommendation task in presence of binary user feedback and cold-start situations, that is, predicting ratings for users who have a few past ratings, and predicting ratings of items that have been rated by a few users. Results show the potential of Linked Open Data-enabled approaches to outperform existing state-of-the-art algorithms.
1. INTRODUCTION
Recently, novel and more accessible forms of information coming from di↵ erent open knowledge sources represent a rapidly growing piece of the big data puzzle.
Over the last years, more and more semantic data are published following the Linked Data principles1, by connecting information referring to geographical locations, people, companies, book, scientific publications, films, music, TV and 1http://www.w3.org/DesignIssues/LinkedData.html
Using open or pooled data from many sources, often
combined and linked with proprietary big data, can help develop
insights di cult to uncover with internal data alone [
On the other hand, the use of a huge amount of interlinked data poses new challenges to recommender systems researchers, who have to find e↵ ective ways to integrate such knowledge into recommendation paradigms.
This paper presents a preliminary investigation in which we propose and evaluate di↵ erent ways of including several kinds of Linked Open Data features in di↵ erent classes of recommendation algorithms. The evaluation is focused on the top-N recommendations task in presence of binary user feedback and cold-start situations.
This paper extends our previous work carried out to
participate to the Linked Open Data-enabled Recommender
Systems challenge3 [
Previous attempts to build recommender systems that
exploit Linked Open Data are presented in [
In that work, the semantics of the DBpedia relations is not
taken into account, di↵erently from the approach described
in [
In [
DBpedia is also used in [
The increasing interest in using Linked Open Data to
create a new breed of content-based recommender systems is
witnessed by the success of the recent Linked Open
Dataenabled Recommender Systems challenge held at the
European Semantic Web Conference (ESWC 2014). The contest
consisted of 3 tasks, namely rating prediction in cold-start
situations, top-N recommendation from binary user
feedback, and diversity. Interestingly, top-N recommendation
from binary user feedback was the task with the highest
number of participants. The best performing approach was
based on an ensemble of algorithms based on popularity,
Vector Space Model, Random Forests, Logistic Regression,
and PageRank, running on a diverse set of semantic features
[
Similarly to the best performing approach, the second best
performing one was based on the same ingredients [
Section 3.1 describes the set of die↵rent features extracted from the Linked Open Data cloud, while Section 3.2 presents di↵erent kinds of recommendation algorithms, i.e. those based on vector space and probabilistic models, those based on the use of classifiers, and graph-based algorithms, which are fed in di↵erent ways by the features extracted from the Linked Open Data cloud. 3.1
The use of Linked Open Data allows to bridge the gap between the need of background data and the challenge to devise novel advanced recommendation strategies.
There are two main approaches to extract Linked Open Data features to represent items: 1. use of the Uniform Resource Identifier (URI)
The first approach directly extracts DBpedia properties for each item by using its Uniform Resource Identifier (URI). URIs are the standard way to identify real-world entities, and allow to define an entry point to DBpedia.
However, DBpedia provides a huge set of properties for each item, hence a proper strategy to select the most valuable ones is necessary. We could manually identify and select a subset of domain-dependent properties, or we could take into account a subset of the most frequent ones.
Referring to the book domain, in which we performed the evaluation, we selected the 10 properties in Table 1, which are both very frequent and representative of the specific domain.
Starting from these properties, further resources could be recursively added. For example, starting from a book, we could retrieve its author through the property http://dbpedia.org/ontology/author and then retrieve and link other resources by the same author, or other genres of works by the same author.
As an example, the resulting representation obtained for the book The Great and Secret Show is provided in Figure 2. The book is linked to its author (Clive Barker ), to the genre (Fantasy literature), and to the Wikipedia categories (British fantasy novels and 1980s fantasy novels). Furthermore, other books by Clive Barker are reported, such as Books of Blood and Mister B. Gone.
The second approach to extract LOD features uses entity linking algorithms to identify a set of Wikipedia concepts occurring in the item description. Next, those Wikipedia concepts can be easily mapped to the corresponding DBpedia nodes. 4http://bnb.data.bl.uk/ 5http://skipforward.opendfki.de/wiki/DBTropes Property http://dbpedia.org/ontology/wikiPageWikiLink http://purl.org/dc/terms/subject http://dbpedia.org/property/genre http://dbpedia.org/property/publisher http://dbpedia.org/ontology/author http://dbpedia.org/property/followedBy http://dbpedia.org/property/precededBy http://dbpedia.org/property/series http://dbpedia.org/property/dewey http://dbpedia.org/ontology/nonFictionSubject Description Link from a Wikipedia page to another Wikipedia page. This property allows to take into account other Wikipedia pages which are somehow related. The topic of a book.
The genre of a book.
The publisher of a book.
The author of a book.
The book followed by a specific book.
The book preceded by a specific book.
The series of a book.
The Dewey Decimal library Classification.
The subject of a non-fiction book (e.g.: history, biography, cookbook, ...).
Several techniques can be adopted, such as Explicit
Semantic Analysis [
In this work we adopt Tagme, that implements an anchor disambiguation algorithm to produce a Wikipedia-based representation of text fragments, where the most relevant concepts occurring in the text are mapped to the Wikipedia articles they refer to. Tagme performs a sort of feature selection by filtering out the noise in text fragments, and its main advantage is the ability to annotate very short texts.
As an example, the resulting representation obtained for the book The Great and Secret Show is provided in Figure 3. Interestingly, the technique is able to associate several concepts which are somehow related to the book, and which could be useful to provide accurate and diverse recommendations, as well.
All these features are used in di↵erent ways by the di↵erent recommendation algorithms presented in the following section. Details are reported in Section 4.2. 3.2
We tested three di↵erent classes of algorithms for generating top-N recommendations, by using several combinations of features extracted from the Linked Open Data cloud. 3.2.1
Most content-based recommender systems rely on simple retrieval models to produce recommendations, such as keyword matching or Vector Space Model (VSM).
VSM emerged as one of the most e↵ective approaches in the area of Information Retrieval, thanks to its good compromise between e↵ectiveness and simplicity. Documents and queries are represented by vectors in an n-dimensional vector space, where n is the number of index terms (words, stems, concepts, etc.).
Formally, each document is represented by a vector of weights, where weights indicate the degree of association between the document and index terms.
Given this representation, documents are ranked by computing the distance between their vector representations and the query vector. Let D = {d1, d2, ..., dN } denote a set of documents or corpus, and T = {t1, t2, ..., tn} be the dictionary, that is to say the set of words in the corpus. T is obtained by applying some standard natural language processing operations, such as tokenization, stopwords removal, and stemming. Each document dj is represented as a vector in a n-dimensional vector space, so dj = {w1j, w2j, ..., dnj}, where wkj is the weight for term tk in document dj. The most common weighting scheme is the TF-IDF (Term FrequencyInverse Document Frequency).
In content-based recommender systems relying on VSM, the query is the user profile, obtained as a combination of the index terms occurring in the items liked by that user, and recommendations are computed by applying a vector similarity measure, such as the cosine coecient, between the user profile and the items to be recommended in the same vector space.
However, VSM is not able to manage either the latent
semantics of each document or the position of the terms
occurring in it. Hence, we proposed an approach able to produce
a lightweight and implicit semantic representation of
documents (items and user profiles). The technique is based on
the distributional hypothesis, according to which “words that
occur in the same contexts tend to have similar meanings”
[
The gist of the technique is presented in [
The use of fine-grained representations of contexts calls
for specific techniques for reducing the dimensionality of
vectors. Besides the classical Latent Semantic Indexing, which
su↵ers of scalability issues, more scalable techniques were
investigated, such as Random Indexing [
Random Indexing in an incremental method which allows to reduce a vector space by projecting the points into a randomly selected subspace of enough high dimensionality. The goal of using eVSM is to compare a vector space representation which adopts very few dimensions for representing items, with respect to a classical VSM.
As an alternative to VSM, we used the BM25 probabilistic
model [
R = X
nt · (↵ + 1) t2 q nt + ↵ · (1 + av|Ig|dl ) · idf (t) (1) nt is frequency of t in the item I, ↵ and are free parameters, avgdl is the average item length, and idf (t) is the IDF of feature t: idf (t) = log
N
df (t) + 0.5 df (t) + 0.5 df (t) is the number of items in which the feature t occurs, N is the cardinality of the collection.
For all the previous models we explicitly managed
negative preferences of users by adopting the vector negation
operator proposed in [
Several works generally rely on the Rocchio algorithm [
Negative relevance feedback is also discussed in [
This removal operation is called vector negation, which is
related to the concept of orthogonality, and it is proposed in
[
The recommendation process can be seen as a binary classification task, in which each item has to be classified as interesting or not with respect to the user preferences.
We learned classifiers using two algorithms, namely
Random Forests (RF) [
RF is an ensemble learning method, combining di↵erent tree predictors built using di↵erent samples of the training data and random subsets of the data features. The class of an item is determined by the majority voting of the classes returned by the individual trees. The use of di↵erent samples of the data from the same distribution and of di↵erent sets of features for learning the individual trees prevent the overfitting.
LR is a supervised learning method for classification which builds a linear model based on a transformed target variable. 3.2.3
We adopted PageRank with Priors, widely used to obtain
an authority score for a node based on the network
connectivity. Di↵erently from PageRank, it is biased towards
the preferences of a specific user, by adopting a non-uniform
personalization vector to assign die↵rent weights to di↵erent
nodes [
In order to run the PageRank, we need to represent data using a graph model. To this purpose, users and items in the dataset are represented as nodes of a graph, while links are represented by the positive users’ feedback. The graph may be enriched in die↵rent ways, for example exploiting entities and relations coming from DBpedia: in this case the whole graph would contain nodes representing users, items, and entities, and edges representing items relevant to users, and relations between entities. This unified representation allows to take into account both collaborative and contentbased features to produce recommendations.
In the classic PageRank, the prior probability assigned to each node is evenly distributed ( 1 , where N is the number of N nodes), while PageRank with Priors is biased towards some nodes, i.e. the preferences of a specific user (see Section 4.2). 4.
The goal of the experiments is to evaluate the contribution of diverse combinations of features, including those extracted from the Linked Open Data cloud, to the accuracy of die↵rent classes of recommendation algorithms.
The experiments that have been carried out try to answer to the following questions: 1. Which is the contribution of the Linked Open Data features to the accuracy of top-N recommendations algorithms, in presence of binary user feedback and cold-start situations? 2. Do the Linked Open Data-enabled approaches outperform existing state-of-the-art recommendation algorithms? 4.1
The dataset used in the experiment is DBbook, coming from the recent Linked-Open Data-enabled Recommender Systems challenge. It contains user preferences retrieved from the Web in the book domain. Each book is mapped to the corresponding DBpedia URI, which can be used to extract features from di↵erent datasets in the Linked Open Data cloud.
The training set released for the top-N recommendation task contains 72,372 binary ratings provided by 6,181 users on 6,733 items. The dataset sparsity is 99.83%, and the distribution of ratings is reported in Table 2.
The test set contains user-item pairs to rank in order to produce a top-5 item recommendation list for each user, to be evaluated using F1@5 accuracy measure. 4.2
Each recommendation algorithm is fed by a diverse set of features.
Besides TAGME and LOD features, algorithms may also use BASIC features, i.e. number of positive, number of negative, and total number of feedbacks provided by users and provided on items, ratio between positive, negative and total number of feedbacks provided by users and provided on items and CONTENT features, obtained by processing book descriptions gathered from Wikipedia. A simple NLP pipeline removes stopwords, and applies stemming. For books not existing in Wikipedia, DBpedia abstracts were processed.
For all the methods, the 5 most popular items are assigned as liked to users with no positive ratings in the training set. Indeed, 5.37 is the average number of positive ratings for each user in the dataset (see Table 2).
Recommender systems relying on VSM and probabilistic framework index items using CONTENT, TAGME and LOD features, and use as query the user profile obtained by combining all the index terms occurring in the items liked by that user.
Items in the test set are ranked by computing the
similarity with the user profile. For VSM and eVSM the cosine
measure is adopted, while Equation 1 is used for the
probabilistic model. According to the literature [
Classifiers based on Random Forests and Logistic
Regression are trained with examples represented using CONTENT,
TAGME and LOD features, and labeled with the binary ratings
provided by users. The value of each feature is the
number of times it occurs in each item, normalized in the [
The LR classifier always includes BASIC features in the training examples, while these did not provide valuable results for RF.
The RF classifier used 1,500 trees to provide a good tradeo↵ between accuracy and eciency.
For Logistic Regression we adopted the implementation provided by Liblinear6, while for Random Forests we adopted the implementation provided by the Weka library7.
Top-N recommendations are produced by ranking items according to the probability of the class.
PageRank with Priors is performed (for each single user) using graphs with di↵erent sets of nodes. Initially, only users, items and links represented by the positive feedback are included; next, we enriched the graph with the 10 properties extracted from DBpedia (see Section 3.1). Then, we ran a second level expansion stage of the graph to retrieve the following additional resources: 1. internal wiki links of the new added nodes 2. more generic categories according to the hierarchy in
DBpedia
5. genres pertaining to the author of the book
This process adds thousands of nodes to the original graph. For this reason, we pruned the graph by removing nodes which are neither users nor books and having a total number of inlinks and outlinks less than 5. This graph eventually consisted of 340,000 nodes and 6 millions links.
The prior probabilities assigned to nodes depend on the users’ preferences, and are assigned according to the following heuristics: 80% of the total weight is evenly distributed among items liked by users (0 assigned to disliked items), 20% is evenly distributed among the remaining nodes. We ran the algorithm with a damping factor set to 0.85.
We adopted the implementation of PageRank provided by the Jung library8.
The PageRank computed for each node is used to rank items in the test set. 4.3
Statistics about users
Statistics about items
Value Classifiers VSM eVSM BM25 s r u o H
The best configuration for eVSM adopts TAGME features alone, and is significantly better than all the configurations but the one combining CONTENT and TAGME features (p = 0.13). This could mean that the entity linking algorithm is able to select the most important features in the book descriptions, while CONTENT features introduce noise.
For BM25, the best configuration with ALL the features significantly outperforms all the others but the one combining CONTENT and LOD features (p = 0.53).
Surprisingly, there is no statistical di↵erence between the best performing configuration for VSM and the best one for BM25.
A final remark is that eVSM performance is not comparable to the other methods, even though it is worth noting that it represents items using very low-dimensional vectors (dimension=500), compared to VSM, which uses vectors whose dimensionality is equal to the number of items (6,733).
Figure 5 presents the results obtained by the classifiers.
We note that Logistic Regression always outperforms Random Forests, and provides better results than the vector space and probabilistic models, regardless the set of adopted features.
The best result using Logistic Regression is obtained with TAGME features alone. This configuration significantly outperforms the one including CONTENT and LOD features (p < 0.05), while it is not di↵erent with respect to the other configurations. This is probably due to the high sparsity of the feature vector used to represent each training example (220,000 features).
Random Forests classifiers outperform eVSM, but they are 0,5600 0,5500 0,5400 10,5300 F 0,5200 0,5100 0,5000 0,5600 0,5500 0,5400 10,5300 F 0,5200 0,5100 0,5000 worse than vector space and probabilistic models. The best result is obtained using ALL features. Since Random Forests classifiers are able to automatically perform feature selection, this was an unexpected result which deserves further investigations.
Finally, Figure 6 presents the results obtained by the PageRank with Priors algorithm.
PageRank with Priors No Content 10 properties 10 properties + 10 properties + expansion stage + expansion stage
pruning PageRank
Execution time
When using PageRank with Priors, we observe the impact of the graph size on both the accuracy and execution time. Starting with a graph not including content information, we observe the worst performance and the lowest execution time (2 hours on an Intel i7 3Ghz 32Gb RAM - the algorithm is performed for each user with di↵erent weights initially assigned to the nodes).
Enriching the graph with the 10 selected DBpedia properties leads to an improvement of accuracy (p < 0.001), and to a 5 hours execution time. Running the expansion stage and pruning of nodes as described in Section 4.2, the time needed to run the algorithm increases to 14 hours and produces a slight accuracy improvement (p < 0.001). Results using the graph with no pruning procedure are not di↵erent from the previous method (p = 0.09), but its time complexity is not acceptable. This call for a more ecient implementation of the algorithm.
To complete the empirical evaluation, we compare the best performing configuration of each algorithm in each class, with some state-of-the-art algorithms.
More specifically, we report the performance of user-touser and item-to-item collaborative filtering, besides two nonpersonalized baselines based on popularity and random recommendations.
Furthermore, we report the results for two algorithms for
top-N recommendations from implicit feedback: an
extension of matrix factorization optimized for Bayesian
Personalized Ranking (BPRMF ) [
Except for SPRank, we used the implementations
available in MyMediaLite 3.10 [
The analysis of results in Figure 7 unveils the diculty
of collaborative filtering algorithms to deal with the high
sparsity of the dataset (99.83%), and with the high number
of users who provided only negative preferences, or more
negative than positive ratings. It is unexpected the
better performance of BPRMF compared to SPRank,
di↵erently from previous results obtained on the MovieLens and
Last.fm datasets [
Overall Comparison 0,5600 0,5500 0,5400 F10,5300 0,5200 0,5100 0,5000
The analysis of the previous results allows to conclude that TAGME and LOD features have the potential to improve the performance of several recommendation algorithms for computing top-N recommendations from binary user feedback.
However, in order to generalize our preliminary results, it is necessary to further investigate: • the e↵ect of die↵rent levels of sparsity on the recommendation accuracy: to this purpose, it is needed to assess the extent to which LOD features are able to improve the performance of recommendation algorithms for die↵rent levels of sparsity • the accuracy on other datasets to generalize our conclusions: further experiments on die↵rent target domains are needed. Indeed, di↵erent item types, such as books, movies, news, songs have die↵rent characteristics which could lead to di↵erent results. Moreover, experiments on a much larger scale are needed • the e↵ect of the selection of domain-specific DBpedia properties to feed the recommendation algorithms: it is needed to assess the e↵ect of the selection of specific sets of properties on the performance of the recommendation algorithms. Indeed, DBpedia contains a huge number of properties, and their selection could have a strong influence on the accuracy of the recommendation methods. Our preliminary experiments leverage 10 DBpedia properties which are both frequent and representative of the specific domain, but a subset of these properties, or a di↵erent set of features could lead to di↵erent results.
As future work, we will study the ee↵ct of enriching the graph-based representation with DBpedia nodes extracted from the Tagme entity linking algorithm.
Indeed, using entity linking to access DBpedia knowledge is innovative and avoids the need of explicitly finding URIs for items, a complex process which may hinder the use of the Linked Open Data. Hence, the use of entity linking algorithms represents a novel way to access the DBpedia knowledge through the analysis of the item descriptions, without exploiting any explicit mapping of items to URIs.
Furthermore, starting from the preliminary evaluation
carried out in [
This work fulfils the research objectives of the project “VINCENTE - A Virtual collective INtelligenCe ENvironment to develop sustainable Technology Entrepreneurship ecosystems” (PON 02 00563 3470993) funded by the Italian Ministry of University and Research (MIUR). 5.