=Paper=
{{Paper
|id=Vol-1245/paper8
|storemode=property
|title=Linked Open Data-enabled Strategies for Top-N Recommendations
|pdfUrl=https://ceur-ws.org/Vol-1245/cbrecsys2014-paper08.pdf
|volume=Vol-1245
|dblpUrl=https://dblp.org/rec/conf/recsys/MustoBLGS14
}}
==Linked Open Data-enabled Strategies for Top-N Recommendations==
https://ceur-ws.org/Vol-1245/cbrecsys2014-paper08.pdf
Linked Open Data-enabled Strategies for
Top-N Recommendations
Cataldo Musto Pierpaolo Basile Pasquale Lops
Dept. of Computer Science Dept. of Computer Science Dept. of Computer Science
Univ. of Bari Aldo Moro, Italy Univ. of Bari Aldo Moro, Italy Univ. of Bari Aldo Moro, Italy
cataldo.musto@uniba.it pierpaolo.basile@uniba.it pasquale.lops@uniba.it
Marco de Gemmis Giovanni Semeraro
Dept. of Computer Science Dept. of Computer Science
Univ. of Bari Aldo Moro, Italy Univ. of Bari Aldo Moro, Italy
marco.degemmis@uniba.it giovanni.semeraro@uniba.it
ABSTRACT radio programs, genes, proteins, drugs, online communities,
The huge amount of interlinked information referring to dif- statistical data, and reviews in a single global data space,
ferent domains, provided by the Linked Open Data (LOD) the Web of Data [2].
initiative, could be e↵ectively exploited by recommender sys- This information, interlinked with each other, forms a
tems to deal with the cold-start and sparsity problems. global graph called Linked Open Data cloud, whose nucleus
In this paper we investigate the contribution of several is represented by DBpedia2 .
features extracted from the Linked Open Data cloud to the A fragment of the Linked Open Data cloud is depicted in
accuracy of di↵erent recommendation algorithms. We focus Figure 1.
on the top-N recommendation task in presence of binary
user feedback and cold-start situations, that is, predicting
ratings for users who have a few past ratings, and predicting
ratings of items that have been rated by a few users.
Results show the potential of Linked Open Data-enabled
approaches to outperform existing state-of-the-art algorithms.
Categories and Subject Descriptors
H.3.3 [Information Systems]: Information Search and Re-
trieval
Keywords
Content-based Recommender Systems; Top-N recommenda-
tions; Implicit Feedback; Linked Open Data; DBpedia
1. INTRODUCTION Figure 1: Fragment of the Linked Open Data cloud
Recently, novel and more accessible forms of information (as of September 2011).
coming from di↵erent open knowledge sources represent a
rapidly growing piece of the big data puzzle. Using open or pooled data from many sources, often com-
Over the last years, more and more semantic data are pub- bined and linked with proprietary big data, can help develop
lished following the Linked Data principles1 , by connecting insights difficult to uncover with internal data alone [4], and
information referring to geographical locations, people, com- can be e↵ectively exploited by recommender systems to deal
panies, book, scientific publications, films, music, TV and with classical problems of cold-start and sparsity.
1
On the other hand, the use of a huge amount of inter-
http://www.w3.org/DesignIssues/LinkedData.html linked data poses new challenges to recommender systems
researchers, who have to find e↵ective ways to integrate such
knowledge into recommendation paradigms.
This paper presents a preliminary investigation in which
we propose and evaluate di↵erent ways of including several
kinds of Linked Open Data features in di↵erent classes of
recommendation algorithms. The evaluation is focused on
Copyright 2014 for the individual papers by the paper’s authors. the top-N recommendations task in presence of binary user
Copying permitted for private and academic purposes. This volume is feedback and cold-start situations.
published and
CBRecSys copyrighted
2014, October 6, by its editors.
2014, Silicon Valley, CA, USA. 2
Copyright
CBRecSys 2014
2014,byOctober
the author(s).
6, 2014, Silicon Valley, CA, USA. http://dbpedia.org
49
This paper extends our previous work carried out to par- leveraged various features sets created from DBpedia. Ad-
ticipate to the Linked Open Data-enabled Recommender ditional Linked Open Data sources were explored, such as
Systems challenge3 [1], by presenting results for new tested British Library Bibliography4 and DBTropes5 , even though
algorithms, along with the various combinations of features. they did not provide meaningful features with respect to
Results show the potential of Linked Open Data-enabled ap- those derived from DBpedia. The results of the individual
proaches to outperform existing state-of-the-art algorithms. recommenders were combined using stacking regression and
rank aggregation using Borda.
2. RELATED WORK
Previous attempts to build recommender systems that ex- 3. METHODOLOGY
ploit Linked Open Data are presented in [17], where a music Section 3.1 describes the set of di↵erent features extracted
recommender system uses DBpedia to compute the Linked from the Linked Open Data cloud, while Section 3.2 presents
Data Semantic Distance, which allows to provide recommen- di↵erent kinds of recommendation algorithms, i.e. those
dations by computing the semantic distance for all artists based on vector space and probabilistic models, those based
referenced in DBpedia. on the use of classifiers, and graph-based algorithms, which
In that work, the semantics of the DBpedia relations is not are fed in di↵erent ways by the features extracted from the
taken into account, di↵erently from the approach described Linked Open Data cloud.
in [5], where properties extracted from DBpedia and Linked-
MDB [12] are exploited to perform a semantic expansion of the 3.1 Features extracted from the Linked Open
item descriptions, suitable for learning user profiles. Data cloud
In [15], DBpedia is used to enrich the playlists extracted The use of Linked Open Data allows to bridge the gap
from a Facebook profile with new related artists. Each between the need of background data and the challenge to
artist in the original playlist is mapped to a DBpedia node, devise novel advanced recommendation strategies.
and other similar artists are selected by taking into account There are two main approaches to extract Linked Open
shared properties, such as the genre and the musical cate- Data features to represent items:
gory of the artist.
DBpedia is also used in [16] to capture the complex rela- 1. use of the Uniform Resource Identifier (URI)
tionships between users, items and entities by extracting the
paths that connect users to items, in order to compute rec- 2. use of entity linking algorithms.
ommendations through a learning to rank algorithm called
SPRank. SPRank is a hybrid recommendation algorithm The first approach directly extracts DBpedia properties for
able to compute top-N item recommendations from implicit each item by using its Uniform Resource Identifier (URI).
feedback, that e↵ectively incorporates ontological knowledge URIs are the standard way to identify real-world entities,
coming from DBpedia (content-based part) with collabora- and allow to define an entry point to DBpedia.
tive user preferences (collaborative part) in a graph-based However, DBpedia provides a huge set of properties for
setting. Starting from the common graph-based represen- each item, hence a proper strategy to select the most valu-
tation of the content and collaborative data models, all the able ones is necessary. We could manually identify and select
paths connecting the user to an item are considered in or- a subset of domain-dependent properties, or we could take
der to have a relevance score for that item. The more paths into account a subset of the most frequent ones.
between a user and an item, the more that item is relevant Referring to the book domain, in which we performed the
to that user. evaluation, we selected the 10 properties in Table 1, which
The increasing interest in using Linked Open Data to cre- are both very frequent and representative of the specific do-
ate a new breed of content-based recommender systems is main.
witnessed by the success of the recent Linked Open Data- Starting from these properties, further resources could be
enabled Recommender Systems challenge held at the Euro- recursively added. For example, starting from a book, we
pean Semantic Web Conference (ESWC 2014). The contest could retrieve its author through the property
consisted of 3 tasks, namely rating prediction in cold-start http://dbpedia.org/ontology/author
situations, top-N recommendation from binary user feed- and then retrieve and link other resources by the same au-
back, and diversity. Interestingly, top-N recommendation thor, or other genres of works by the same author.
from binary user feedback was the task with the highest As an example, the resulting representation obtained for
number of participants. The best performing approach was the book The Great and Secret Show is provided in Figure
based on an ensemble of algorithms based on popularity, 2. The book is linked to its author (Clive Barker ), to the
Vector Space Model, Random Forests, Logistic Regression, genre (Fantasy literature), and to the Wikipedia categories
and PageRank, running on a diverse set of semantic features (British fantasy novels and 1980s fantasy novels). Further-
[1]. The performance of the single methods were aggregated more, other books by Clive Barker are reported, such as
using the Borda count aggregation strategy. Most of the Books of Blood and Mister B. Gone.
techniques used in the contest are presented in this paper. The second approach to extract LOD features uses entity
Similarly to the best performing approach, the second best linking algorithms to identify a set of Wikipedia concepts
performing one was based on the same ingredients [18]. In- occurring in the item description. Next, those Wikipedia
deed, it combined di↵erent base recommenders, such as col- concepts can be easily mapped to the corresponding DBpedia
laborative and content-based ones, with a non-personalized nodes.
recommender based on popularity. Content-based strategies 4
http://bnb.data.bl.uk/
3 5
challenges.2014.eswc-conferences.org/index.php/RecSys http://skipforward.opendfki.de/wiki/DBTropes
49
Property Description Frequency
http://dbpedia.org/ontology/wikiPageWikiLink Link from a Wikipedia page to another Wikipedia 523,321
page. This property allows to take into account
other Wikipedia pages which are somehow related.
http://purl.org/dc/terms/subject The topic of a book. 38,627
http://dbpedia.org/property/genre The genre of a book. 12,488
http://dbpedia.org/property/publisher The publisher of a book. 9,798
http://dbpedia.org/ontology/author The author of a book. 8,669
http://dbpedia.org/property/followedBy The book followed by a specific book. 6,351
http://dbpedia.org/property/precededBy The book preceded by a specific book. 5,978
http://dbpedia.org/property/series The series of a book. 3,196
http://dbpedia.org/property/dewey The Dewey Decimal library Classification. 1,930
http://dbpedia.org/ontology/nonFictionSubject The subject of a non-fiction book 966
(e.g.: history, biography, cookbook, ...).
Table 1: DBpedia properties selected for the book domain.
Books_of
_Blood
author
Clive_ Mister_
Barker author B._Gone
author Figure 3: Tagme representation of the book The
Great and Secret Show by Clive Barker.
The_
Great_ genre Fantasy_
and_ literature All these features are used in di↵erent ways by the di↵er-
Secret_ ent recommendation algorithms presented in the following
show section. Details are reported in Section 4.2.
subject subject 3.2 Recommendation Algorithms
precededBy We tested three di↵erent classes of algorithms for generat-
ing top-N recommendations, by using several combinations
British_ 1980s_
fantasy_ fantasy_ of features extracted from the Linked Open Data cloud.
novels novels Everville
3.2.1 Algorithms based on the Vector Space and Prob-
abilistic Models
Most content-based recommender systems rely on simple
retrieval models to produce recommendations, such as key-
Figure 2: Properties of the book The Great and Se- word matching or Vector Space Model (VSM).
cret Show by Clive Barker. VSM emerged as one of the most e↵ective approaches in
the area of Information Retrieval, thanks to its good com-
promise between e↵ectiveness and simplicity. Documents
and queries are represented by vectors in an n-dimensional
Several techniques can be adopted, such as Explicit Se- vector space, where n is the number of index terms (words,
mantic Analysis [8] or Tagme [7]. stems, concepts, etc.).
In this work we adopt Tagme, that implements an anchor Formally, each document is represented by a vector of
disambiguation algorithm to produce a Wikipedia-based rep- weights, where weights indicate the degree of association
resentation of text fragments, where the most relevant con- between the document and index terms.
cepts occurring in the text are mapped to the Wikipedia Given this representation, documents are ranked by com-
articles they refer to. Tagme performs a sort of feature se- puting the distance between their vector representations and
lection by filtering out the noise in text fragments, and its the query vector. Let D = {d1 , d2 , ..., dN } denote a set of
main advantage is the ability to annotate very short texts. documents or corpus, and T = {t1 , t2 , ..., tn } be the dictio-
As an example, the resulting representation obtained for nary, that is to say the set of words in the corpus. T is
the book The Great and Secret Show is provided in Figure obtained by applying some standard natural language pro-
3. Interestingly, the technique is able to associate several cessing operations, such as tokenization, stopwords removal,
concepts which are somehow related to the book, and which and stemming. Each document dj is represented as a vector
could be useful to provide accurate and diverse recommen- in a n-dimensional vector space, so dj = {w1j , w2j , ..., dnj },
dations, as well. where wkj is the weight for term tk in document dj . The
50
most common weighting scheme is the TF-IDF (Term Frequency- Several works generally rely on the Rocchio algorithm [21]
Inverse Document Frequency). to incrementally refine the user profiles by exploiting positive
In content-based recommender systems relying on VSM, and negative feedback provided by users, even though the
the query is the user profile, obtained as a combination of method needs an extensive tuning of parameters for being
the index terms occurring in the items liked by that user, e↵ective.
and recommendations are computed by applying a vector Negative relevance feedback is also discussed in [6], in
similarity measure, such as the cosine coefficient, between which the idea of representing negation by subtracting an
the user profile and the items to be recommended in the unwanted vector from a query emerged, even if nothing
same vector space. about how much to subtract is stated. Hence, vector nega-
However, VSM is not able to manage either the latent se- tion is built on the idea of subtracting exactly the right
mantics of each document or the position of the terms occur- amount to make the unwanted vector irrelevant to the re-
ring in it. Hence, we proposed an approach able to produce sults we obtain.
a lightweight and implicit semantic representation of docu- This removal operation is called vector negation, which is
ments (items and user profiles). The technique is based on related to the concept of orthogonality, and it is proposed in
the distributional hypothesis, according to which “words that [23].
occur in the same contexts tend to have similar meanings”
[11]. This means that the meaning of a word is inferred by 3.2.2 Algorithms based on Classifiers
analyzing its usage in large corpora of textual documents, The recommendation process can be seen as a binary clas-
hence words are semantically similar to the extent that they sification task, in which each item has to be classified as
share contexts. interesting or not with respect to the user preferences.
The gist of the technique is presented in [14], in which a We learned classifiers using two algorithms, namely Ran-
novel content-based recommendation framework, called en- dom Forests (RF) [3] and Logistic Regression (LR).
hanced Vector Space Model (eVSM), is described. eVSM RF is an ensemble learning method, combining di↵erent
adopts a latent semantic representation of items in terms tree predictors built using di↵erent samples of the training
of contexts, i.e. a term-context matrix is adopted, instead data and random subsets of the data features. The class of
of the classical term-document matrix adopted in the VSM. an item is determined by the majority voting of the classes
The advantage is that the context can be adapted to the spe- returned by the individual trees. The use of di↵erent sam-
cific granularity level of the representation required by the ples of the data from the same distribution and of di↵erent
application: for example, given a word, its context could sets of features for learning the individual trees prevent the
be either a single word it co-occurs with, a sentence, or the overfitting.
whole document. LR is a supervised learning method for classification which
The use of fine-grained representations of contexts calls builds a linear model based on a transformed target variable.
for specific techniques for reducing the dimensionality of vec-
tors. Besides the classical Latent Semantic Indexing, which 3.2.3 Graph-based Algorithms
su↵ers of scalability issues, more scalable techniques were We adopted PageRank with Priors, widely used to obtain
investigated, such as Random Indexing [22], adopted in the an authority score for a node based on the network con-
eVSM model. nectivity. Di↵erently from PageRank, it is biased towards
Random Indexing in an incremental method which allows the preferences of a specific user, by adopting a non-uniform
to reduce a vector space by projecting the points into a ran- personalization vector to assign di↵erent weights to di↵erent
domly selected subspace of enough high dimensionality. The nodes [13].
goal of using eVSM is to compare a vector space represen- In order to run the PageRank, we need to represent data
tation which adopts very few dimensions for representing using a graph model. To this purpose, users and items in
items, with respect to a classical VSM. the dataset are represented as nodes of a graph, while links
As an alternative to VSM, we used the BM25 probabilistic are represented by the positive users’ feedback. The graph
model [19], one of the most dominant retrieval paradigm may be enriched in di↵erent ways, for example exploiting
today. The ranking function for matching a query q (user entities and relations coming from DBpedia: in this case the
profile) and an item I is: whole graph would contain nodes representing users, items,
and entities, and edges representing items relevant to users,
X nt · (↵ + 1) and relations between entities. This unified representation
R= |I|
· idf (t) (1) allows to take into account both collaborative and content-
t2q nt + ↵ · (1 + avgdl
)
based features to produce recommendations.
nt is frequency of t in the item I, ↵ and are free parame- In the classic PageRank, the prior probability assigned to
ters, avgdl is the average item length, and idf (t) is the IDF each node is evenly distributed ( N1 , where N is the number of
of feature t: nodes), while PageRank with Priors is biased towards some
nodes, i.e. the preferences of a specific user (see Section 4.2).
N df (t) + 0.5
idf (t) = log (2)
df (t) + 0.5 4. EXPERIMENTAL EVALUATION
df (t) is the number of items in which the feature t occurs, The goal of the experiments is to evaluate the contribu-
N is the cardinality of the collection. tion of diverse combinations of features, including those ex-
For all the previous models we explicitly managed neg- tracted from the Linked Open Data cloud, to the accuracy
ative preferences of users by adopting the vector negation of di↵erent classes of recommendation algorithms.
operator proposed in [23], based on the concept of orthogo- The experiments that have been carried out try to answer
nality between vectors. to the following questions:
51
1. Which is the contribution of the Linked Open Data training examples, while these did not provide valuable re-
features to the accuracy of top-N recommendations sults for RF.
algorithms, in presence of binary user feedback and The RF classifier used 1,500 trees to provide a good trade-
cold-start situations? o↵ between accuracy and efficiency.
For Logistic Regression we adopted the implementation
2. Do the Linked Open Data-enabled approaches out- provided by Liblinear6 , while for Random Forests we adopted
perform existing state-of-the-art recommendation al- the implementation provided by the Weka library7 .
gorithms? Top-N recommendations are produced by ranking items
according to the probability of the class.
4.1 Dataset
The dataset used in the experiment is DBbook, coming Graph-based Algorithms.
from the recent Linked-Open Data-enabled Recommender PageRank with Priors is performed (for each single user)
Systems challenge. It contains user preferences retrieved using graphs with di↵erent sets of nodes. Initially, only
from the Web in the book domain. Each book is mapped users, items and links represented by the positive feedback
to the corresponding DBpedia URI, which can be used to are included; next, we enriched the graph with the 10 prop-
extract features from di↵erent datasets in the Linked Open erties extracted from DBpedia (see Section 3.1). Then, we
Data cloud. ran a second level expansion stage of the graph to retrieve
The training set released for the top-N recommendation the following additional resources:
task contains 72,372 binary ratings provided by 6,181 users
on 6,733 items. The dataset sparsity is 99.83%, and the 1. internal wiki links of the new added nodes
distribution of ratings is reported in Table 2.
The test set contains user-item pairs to rank in order to 2. more generic categories according to the hierarchy in
produce a top-5 item recommendation list for each user, to DBpedia
be evaluated using F1@5 accuracy measure. 3. resources of the same category
4.2 Experimental setup 4. resources of the same genre
Each recommendation algorithm is fed by a diverse set of
5. genres pertaining to the author of the book
features.
Besides TAGME and LOD features, algorithms may also use 6. resources written by the author
BASIC features, i.e. number of positive, number of nega-
tive, and total number of feedbacks provided by users and 7. genres of the series the book belongs to.
provided on items, ratio between positive, negative and to-
tal number of feedbacks provided by users and provided on This process adds thousands of nodes to the original graph.
items and CONTENT features, obtained by processing book de- For this reason, we pruned the graph by removing nodes
scriptions gathered from Wikipedia. A simple NLP pipeline which are neither users nor books and having a total num-
removes stopwords, and applies stemming. For books not ber of inlinks and outlinks less than 5. This graph eventually
existing in Wikipedia, DBpedia abstracts were processed. consisted of 340,000 nodes and 6 millions links.
For all the methods, the 5 most popular items are assigned The prior probabilities assigned to nodes depend on the
as liked to users with no positive ratings in the training set. users’ preferences, and are assigned according to the follow-
Indeed, 5.37 is the average number of positive ratings for ing heuristics: 80% of the total weight is evenly distributed
each user in the dataset (see Table 2). among items liked by users (0 assigned to disliked items),
20% is evenly distributed among the remaining nodes. We
Algorithms based on the Vector Space and Probabilis- ran the algorithm with a damping factor set to 0.85.
tic Models. We adopted the implementation of PageRank provided by
Recommender systems relying on VSM and probabilistic the Jung library8 .
framework index items using CONTENT, TAGME and LOD fea- The PageRank computed for each node is used to rank
tures, and use as query the user profile obtained by combin- items in the test set.
ing all the index terms occurring in the items liked by that
user.
4.3 Results
Items in the test set are ranked by computing the simi- Figure 4 shows the results of VSM and probabilistic mod-
larity with the user profile. For VSM and eVSM the cosine els. The paired t-test is used for testing the significance.
measure is adopted, while Equation 1 is used for the proba- The first interesting outcome is that the worst configu-
bilistic model. According to the literature [20], parameters rations are always obtained using LOD data alone, and the
↵ and are set to 1.6 and 0.75, respectively. best ones always contain TAGME features. In detail, the best
VSM configuration is obtained combining TAGME and LOD
features, which is significantly better than using LOD fea-
Algorithms based on Classifiers.
tures alone (p < 0.0001) and CONTENT alone (p < 0.05). The
Classifiers based on Random Forests and Logistic Regres-
combination of CONTENT and LOD features outperforms the
sion are trained with examples represented using CONTENT,
models using just CONTENT (p < 0.01) or just LOD features
TAGME and LOD features, and labeled with the binary ratings
(p < 0.001).
provided by users. The value of each feature is the num-
6
ber of times it occurs in each item, normalized in the [0,1] www.csie.ntu.edu.tw/˜cjlin/liblinear/
7
interval. www.cs.waikato.ac.nz/ml/weka/
8
The LR classifier always includes BASIC features in the jung.sourceforge.net
52
Property Value
Statistics about users
Avg. ratings provided by users 11.71 (5.37 positive, 6.34 negative)
# of users who provided only negative ratings 520 (8.41%)
# of users having a number of positive ratings below the avg. 3,804 (61.54%)
# of users having more negative than positive ratings 3,343 (54.09%)
Statistics about items
Avg. ratings received by items 10.75 (4.93 positive, 5.82 negative)
# of items with no positive ratings 1,839 (27.31%)
# of items having a number of positive ratings below the avg. 6,447 (95.75%)
# of items having more negative than positive ratings 4,046 (60.09%)
Table 2: Distribution of ratings for the DBbook dataset.
VectorSpaceandProbabilisticModels Classifiers
0,5600 0,5600
0,5500 0,5500
0,5400 0,5400
F1
F1
0,5300 0,5300
VSM
0,5200 0,5200 LR
eVSM
0,5100 0,5100 RF
BM25
0,5000 0,5000
Figure 4: Results of the VSM and probabilistic mod- Figure 5: Results of the classifiers using di↵erent
els using di↵erent combinations of features. combinations of features.
worse than vector space and probabilistic models. The best
The best configuration for eVSM adopts TAGME features
result is obtained using ALL features. Since Random Forests
alone, and is significantly better than all the configurations
classifiers are able to automatically perform feature selec-
but the one combining CONTENT and TAGME features (p =
tion, this was an unexpected result which deserves further
0.13). This could mean that the entity linking algorithm
investigations.
is able to select the most important features in the book
Finally, Figure 6 presents the results obtained by the PageR-
descriptions, while CONTENT features introduce noise.
ank with Priors algorithm.
For BM25, the best configuration with ALL the features
significantly outperforms all the others but the one combin-
ing CONTENT and LOD features (p = 0.53). PageRankwithPriors
Surprisingly, there is no statistical di↵erence between the 0,5600 140
best performing configuration for VSM and the best one for 0,5500 120
BM25. 0,5400
100
Hours
A final remark is that eVSM performance is not compara- 80
F1
0,5300
ble to the other methods, even though it is worth noting that 0,5200
60
40
it represents items using very low-dimensional vectors (di- 0,5100 20
mension=500), compared to VSM, which uses vectors whose 0,5000 0
dimensionality is equal to the number of items (6,733). NoContent 10properties 10properties+ 10properties+
expansionstage+ expansionstage
Figure 5 presents the results obtained by the classifiers. pruning
We note that Logistic Regression always outperforms Ran-
PageRank Executiontime
dom Forests, and provides better results than the vector
space and probabilistic models, regardless the set of adopted
features.
The best result using Logistic Regression is obtained with Figure 6: Results of the PageRank with Priors using
TAGME features alone. This configuration significantly out- di↵erent combinations of features.
performs the one including CONTENT and LOD features (p <
0.05), while it is not di↵erent with respect to the other con- When using PageRank with Priors, we observe the impact
figurations. This is probably due to the high sparsity of of the graph size on both the accuracy and execution time.
the feature vector used to represent each training example Starting with a graph not including content information, we
(220,000 features). observe the worst performance and the lowest execution time
Random Forests classifiers outperform eVSM, but they are (2 hours on an Intel i7 3Ghz 32Gb RAM - the algorithm
53
is performed for each user with di↵erent weights initially However, in order to generalize our preliminary results, it
assigned to the nodes). is necessary to further investigate:
Enriching the graph with the 10 selected DBpedia proper-
ties leads to an improvement of accuracy (p < 0.001), and to • the e↵ect of di↵erent levels of sparsity on the recom-
a 5 hours execution time. Running the expansion stage and mendation accuracy: to this purpose, it is needed to
pruning of nodes as described in Section 4.2, the time needed assess the extent to which LOD features are able to im-
to run the algorithm increases to 14 hours and produces a prove the performance of recommendation algorithms
slight accuracy improvement (p < 0.001). Results using the for di↵erent levels of sparsity
graph with no pruning procedure are not di↵erent from the • the accuracy on other datasets to generalize our con-
previous method (p = 0.09), but its time complexity is not clusions: further experiments on di↵erent target do-
acceptable. This call for a more efficient implementation of mains are needed. Indeed, di↵erent item types, such
the algorithm. as books, movies, news, songs have di↵erent character-
To complete the empirical evaluation, we compare the best istics which could lead to di↵erent results. Moreover,
performing configuration of each algorithm in each class, experiments on a much larger scale are needed
with some state-of-the-art algorithms.
More specifically, we report the performance of user-to- • the e↵ect of the selection of domain-specific DBpedia
user and item-to-item collaborative filtering, besides two non- properties to feed the recommendation algorithms: it
personalized baselines based on popularity and random rec- is needed to assess the e↵ect of the selection of spe-
ommendations. cific sets of properties on the performance of the rec-
Furthermore, we report the results for two algorithms for ommendation algorithms. Indeed, DBpedia contains a
top-N recommendations from implicit feedback: an exten- huge number of properties, and their selection could
sion of matrix factorization optimized for Bayesian Personal- have a strong influence on the accuracy of the rec-
ized Ranking (BPRMF ) [9] and SPRank [16], able to exploit ommendation methods. Our preliminary experiments
LInked Open Data knowledge bases to compute accurate leverage 10 DBpedia properties which are both frequent
recommendations. and representative of the specific domain, but a subset
Except for SPRank, we used the implementations avail- of these properties, or a di↵erent set of features could
able in MyMediaLite 3.10 [10], using the default parameters. lead to di↵erent results.
The analysis of results in Figure 7 unveils the difficulty
of collaborative filtering algorithms to deal with the high As future work, we will study the e↵ect of enriching the
sparsity of the dataset (99.83%), and with the high number graph-based representation with DBpedia nodes extracted
of users who provided only negative preferences, or more from the Tagme entity linking algorithm.
negative than positive ratings. It is unexpected the bet- Indeed, using entity linking to access DBpedia knowledge
ter performance of BPRMF compared to SPRank, di↵er- is innovative and avoids the need of explicitly finding URIs
ently from previous results obtained on the MovieLens and for items, a complex process which may hinder the use of
Last.fm datasets [16]. It is also surprising the better perfor- the Linked Open Data. Hence, the use of entity linking algo-
mance of simple algorithms based on the vector space and rithms represents a novel way to access the DBpedia knowl-
probabilistic models with respect to matrix factorization. edge through the analysis of the item descriptions, without
exploiting any explicit mapping of items to URIs.
Furthermore, starting from the preliminary evaluation car-
OverallComparison ried out in [1], we will thoroughly investigate the potential
0,5600
of using the wealth of relations of LOD features to produce
not only accurate, but also diversified recommendation lists.
0,5500
0,5400 Acknowledgments
This work fulfils the research objectives of the project “VIN-
F1
0,5300
0,5200 CENTE - A Virtual collective INtelligenCe ENvironment
0,5100
to develop sustainable Technology Entrepreneurship ecosys-
tems” (PON 02 00563 3470993) funded by the Italian Min-
0,5000
istry of University and Research (MIUR).
5. REFERENCES
[1] P. Basile, C. Musto, M. de Gemmis, P. Lops,
F. Narducci, and G. Semeraro. Aggregation strategies
Figure 7: Comparison with other state-of-the-art for linked open data-enabled recommender systems. In
approaches. European Semantic Web Conference (satellite Events),
2014.
[2] C. Bizer. The emerging web of linked data. IEEE
4.4 Discussion Intelligent Systems, 24(5):87–92, 2009.
The analysis of the previous results allows to conclude [3] L. Breiman. Random forests. Machine Learning,
that TAGME and LOD features have the potential to improve 45(1):5–32, 2001.
the performance of several recommendation algorithms for [4] M. Chui, J. Manyika, and S. V. Kuiken. What
computing top-N recommendations from binary user feed- executives should know about open data. McKinsey
back. Quarterly, January 2014, 2014.
54
[5] T. Di Noia, R. Mirizzi, V. C. Ostuni, and D. Romito. data. In European Semantic Web Conference (Satellite
Exploiting the web of data in model-based Events), 2014.
recommender systems. In P. Cunningham, N. J. [19] S. E. Robertson, S. Walker, M. H. Beaulieu, A. Gull,
Hurley, I. Guy, and S. S. Anand, editors, Proceedings and M. Lau. Okapi at TREC. In Text REtrieval
of the ACM Conference on Recommender Systems ’12, Conference, pages 21–30, 1992.
pages 253–256. ACM, 2012. [20] S. E. Robertson and H. Zaragoza. The probabilistic
[6] M. D. Dunlop. The e↵ect of accessing nonmatching relevance framework: Bm25 and beyond. Foundations
documents on relevance feedback. ACM Trans. Inf. and Trends in Information Retrieval, 3(4):333–389,
Syst., 15:137–153, 1997. 2009.
[7] P. Ferragina and U. Scaiella. Fast and Accurate [21] J. Rocchio. Relevance Feedback Information Retrieval.
Annotation of Short Texts with Wikipedia Pages. In Gerald Salton, editor, The SMART retrieval system
IEEE Software, 29(1):70–75, 2012. - experiments in automated document processing, pages
[8] E. Gabrilovich and S. Markovitch. Wikipedia-based 313–323. Prentice-Hall, Englewood Cli↵s, NJ, 1971.
semantic interpretation for natural language [22] M. Sahlgren. An introduction to random indexing. In
processing. Journal of Artificial Intelligence Research Proc. of the Methods and Applications of Semantic
(JAIR), 34:443–498, 2009. Indexing Workshop at the 7th Int. Conf. on
[9] Z. Gantner, L. Drumond, C. Freudenthaler, S. Rendle, Terminology and Knowledge Engineering, 2005.
and L. Schmidt-Thieme. Learning attribute-to-feature [23] D. Widdows. Orthogonal negation in vector spaces for
mappings for cold-start recommendations. In G. I. modelling word-meanings and document retrieval. In
Webb, B. Liu, C. Zhang, D. Gunopulos, and X. Wu, Proceedings of the 41st Annual Meeting of the
editors, 10th IEEE International Conference on Data Association for Computational Linguistics, pages
Mining, pages 176–185. IEEE Computer Society, 2010. 136–143, 2003.
[10] Z. Gantner, S. Rendle, C. Freudenthaler, and
L. Schmidt-Thieme. Mymedialite: a free recommender
system library. In B. Mobasher, R. D. Burke,
D. Jannach, and G. Adomavicius, editors, Proceedings
of the ACM Conference on Recommender Systems ’11,
pages 305–308. ACM, 2011.
[11] Z. S. Harris. Mathematical Structures of Language.
Interscience, New York 1968.
”
[12] O. Hassanzadeh and M. P. Consens. Linked movie
data base. In C.Bizer, T. Heath, T. Berners-Lee, and
K. Idehen, editors, Proceedings of the WWW2009
Workshop on Linked Data on the Web, LDOW 2009,
volume 538 of CEUR Workshop Proceedings.
CEUR-WS.org, 2009.
[13] T. H. Haveliwala. Topic-Sensitive PageRank: A
Context-Sensitive Ranking Algorithm for Web Search.
IEEE Trans. Knowl. Data Eng., 15(4):784–796, 2003.
[14] C. Musto. Enhanced vector space models for
content-based recommender systems. In Proceedings of
the ACM Conference on Recommneder Systems ’10,
pages 361–364. ACM, 2010.
[15] C. Musto, G. Semeraro, P. Lops, M. de Gemmis, and
F. Narducci. Leveraging social media sources to
generate personalized music playlists. In C. Huemer
and P. Lops, editors, E-Commerce and Web
Technologies - 13th International Conference, EC-Web
2012, volume 123 of Lecture Notes in Business
Information Processing, pages 112–123. Springer, 2012.
[16] V. C. Ostuni, T. Di Noia, E. Di Sciascio, and
R. Mirizzi. Top-n recommendations from implicit
feedback leveraging linked open data. In Q. Yang,
I. King, Q. Li, P. Pu, and G. Karypis, editors,
Proceedings of the ACM Conference on Recommender
Systems ’13, pages 85–92. ACM, 2013.
[17] A. Passant. dbrec - Music Recommendations Using
DBpedia. In International Semantic Web Conference,
Revised Papers, volume 6497 of LNCS, pages 209–224.
Springer, 2010.
[18] P. Ristoski, E. L. Mencia, and H. Paulheim. A hybrid
multi-strategy recommender system using linked open
55