Model-based approaches to recommendation have proven to be very accurate. Unfortunately, exploiting a latent space we miss references to the actual semantics of recommended items. In this extended abstract, we show how to initialize latent factors in Factorization Machines by using semantic features coming from a knowledge graph in order to train an interpretable model. Finally, we introduce and evaluate the semantic accuracy and robustness for the knowledge-aware interpretability of the model.
Transparency and interpretability of predictive models are gaining
momentum since they have been recognized as a key element in the
next generation of recommendation algorithms. When equipped
with interpretability of recommendation results, a system ceases to
be just a black-box and users are more willing to extensively exploit
the predictions [
In [
Í ⟨ρ,ω⟩∈F | { ⟨ρ, ω ⟩ | ⟨i, ρ, ω ⟩ ∈ K G } | (1) Analogously, when we have a set U of users, we may represent them using the features describing the items they enjoyed in the past. We use f to denote a feature ⟨ρ, ω⟩ ∈ F . Given a user u, if we denote with Iu the set of the items enjoyed by u, we may introduce the vector u = [v(u,f1) . . . , v(u,f|F |)], where v(u,f ) is the generic element computed as: v(u,f ) = | {i | i ∈ I u and v(i,f ) , 0} | Given the vectors uj , with j ∈ [1 . . . |U |], and ip , with p ∈ [1 . . . |I |], we build a matrix V ∈ Rn×|F | , where n = |U | + |I |: so the first |U | rows have a one to one mapping with uj while the last ones correspond to ip . In second degree Factorization Machines models the score is computed as: yˆ(xui) = w0 + Õn wj · xj + Õn Õn j=1 j=1 p=j+1
k xj · xp · Õ v(j,f ) · v(p,f ) f =1 (2) We may see that, for each x, the term Ínj=1 Ípn=j+1 xj ·xp ·Íkf=1 v(j,f )· v(p,f ) is non-zero, i.e., when both xj and xp are equal to 1. In a recommendation scenario, this happens when there is an interaction between a user and an item. Moreover, the summation Ík
f =1 v(j,f ) · v(p,f ) represents the dot product between two vectors: vj and vp with a size equal to k. Hence, vj represents a latent representation of a user, vp that of an item within the same latent space, and their interaction is evaluated through their dot product.
In order to inject the knowledge coming from K G into kaHFM,
we set k = |F | in Equation 2. In other words, we impose a number
of latent factors equal to the number of features describing all the
items in our catalog. Since we formulated our problem as a
topN recommendation task, kaHFM can be trained using a learning
to rank approach like Bayesian Personalized Ranking Criterion
(BPR)[
We evaluated the performance of our method on two well-known
datasets for recommender systems belonging to movies domain:
Yahoo!Movies12, and Facebook Movies3. Experiments were
conducted adopting the "All Unrated Items" protocol, and an Hold-Out
80-20 temporal split [
Accuracy Evaluation. The goal of this evaluation is to assess
if the controlled injection of Linked Data positively afects the
training of F M. We compared kaHFM5 w.r.t. a canonical 2 degree
F M optimized via BPR (BPR-FM). In order to preserve the
expressiveness of the model, we used the same number of hidden factors
as kaHFM. Since we use items similarity in the last step of our
approach, we compared kaHFM against an Attribute Based Item-kNN
(ABItem-kNN) algorithm, where each item is represented as a vector
of weights, computed through a TF-IDF model. We also compared
kaHFM against Item-kNN, and User-kNN based on Cosine Similarity,
Most-Popular, and a knowledge-graph-based V SM adopting the
representation formalted in [
Facebook Movies Yahoo!Movies Categorical Setting (CS) Precision@10 Precision@10 nDCG@10 ABItem-kNN 0.0173∗ 0.0421∗ 0.1174∗ BPR-FM 0.0158∗ 0.0189∗ 0.0344∗ MostPopular 0.0118∗ 0.0154∗ 0.0271∗ ItemKnn 0.0262∗ 0.0203∗ 0.0427∗ UserKnn 0.0168∗ 0.0231∗ 0.0474∗ VSM 0.0185∗ 0.0385∗ 0.1129∗ kaHFM 0.0296 0.0524 0.1399 Table 1: Accuracy results for Facebook Movies, and
relevance threshold of 4 over 5 stars. results. We highlight in bold the best result while we underline the second one. Statistically significant diferences in performance are denoted with a ∗ mark considering Student’s paired t-test with a 0.05 level.
Semantic Accuracy. The main idea behind Semantic Accuracy is to evaluate, given an item i, how well kaHFM is able to return its original features available in the computed top-K list vi . In other words, subset i represented by F i = { f1i , . . . , f mi , . . . f Mi }, with F i ⊆ F , we check if the values in vi , corresponding to fm,i ∈ F i , are higher than those corresponding to f < F i . For the set of M features initially describing i we see how many of them appear in the set top(vi , M) representing the top-M features in vi . We then normalize this number by the size of F i and average on all the items within the catalog I . Table 2 shows the results for SA@nM with n ∈ {1, 2, 3, 4, 5} and M = 10, and evaluated the number of ground features available in the top-nM elements of vi for each dataset.
Generative Robustness. To check if kaHFM promotes important features for an item i we proposed a new measure: Generative 1Yahoo! Webscope dataset ydata-ymovies-user-movie-ratings-content-v1_0 2http://research.yahoo.com/Academic_Relations 3https://2015.eswc-conferences.org/program/semwebeval.html 4https://github.com/sisinflab/LinkedDatasets/ 5https://github.com/sisinflab/HybridFactorizationMachines/
Semantics Accuracy SA@M SA@2M SA@3M SA@4M SA@5M F.A. Yahoo!Movies 0.847 0.863 0.865 0.868 0.873 12.143 Facebook Movies 0.864 0.883 0.889 0.894 0.899 12.856 Table 2: Semantics Accuracy results for diferent values of
Robustness. We suppose that a particular feature ⟨ρ, ω⟩ is useful to describe an item i but the corresponding triple ⟨i, ρ, ω⟩ is not represented in the knowledge graph. In case kaHFM was robust in generating weights for unknown features, it should discover the importance of that feature and modify its value to make it enter the Top-K features in vi . Starting from this observation, the idea to measure robustness is then to “forget” a triple involving i and check if kaHFM can generate it. Given a catalog I , we may then define the Robustness for 1 removed feature @M (1-Rob@M) as the number of items for which the removed feature is in T op − M after training. Similarly to SA@nM, we may define 1-Rob@nM. Table 2 1-Robustness 1-Rob@M1-Rob@2M1-Rob@3M1-Rob@4M1-Rob@5M F.A. Yahoo!Movies 0.487 0.645 0.713 0.756 0.793 12.143 Facebook Movies 0.821 0.945 0.970 0.980 0.984 12.856 Table 3: 1-Robustness for diferent values of M. Column F.A. denotes the Feature Average number per item. showed that kaHFM was able to guess 10 on 12 diferent features for Yahoo!Movies. In this experiment, we remove one of the ten features (thus, based on Table 2, kaHFM will guess an average of 10 − 1 = 9 features). Since the number of features is 12 we have 3 remaining "slots". In Table 3, we measure how often kaHFM is able to guess the removed feature in these "slots".
We have proposed an interpretable method for recommendation scenario, kaHFM, in which we bind the meaning of latent factors for a Factorization machine to data coming from a knowledge graph. We considered Categorical information coming from DBpedia and we have shown that the generated recommendations are more precise and personalized on two diferent publicly available datasets. We showed that the computed features are semantically meaningful, and the model is robust regarding computed features. In the future we want to test the kaHFM performance in classical Information Retrieval, and knowledge graph completion tasks.