In this work we propose Ask Me Any Rating (AMAR), a novel content-based recommender system based on deep neural networks which is able to produce top-N recommendations leveraging user and item embeddings which are learnt from textual information describing the items. A comprehensive experimental evaluation conducted on stateof-the-art datasets showed a significant improvement over all the baselines taken into account.
Internet users have access to a huge amount of information, which grows exponentially every day. They are forced to dive among a lot of different information sources from which they have to choose the most appealing contents. Recommender systems can help users to solve the information overload problem because they are able to filter contents according to user preferences. Contentbased recommender systems need to effectively represent user profiles and items to recommend items similar to those a given user has liked in the past.
In recent years, Deep Learning techniques have proved to be particularly
effective in different machine learning fields such as computer vision, language
modeling and speech recognition, reaching state-of-the-art performance. Deep
Learning models learn a hierarchy of levels of representations from data by using
multiple processing layers [
In this work, we present a novel deep neural network model called Ask Me Any Rating (AMAR) which exploits Recurrent Neural Networks (RNNs) to jointly learn a representation for user preferences and items to be recommended and generates a ranked list of items which may be of interest for a given user. Up to our knowledge, this is the first try to use Deep Learning models for content-based recommender systems in a top-N recommendation task.
The paper is organized as follows: section 2 introduces basic terminology and depicts the AMAR architecture. Next, in section 3 we report details about the experimental evaluation conducted to establish the quality of the recommendations generated by this model over different baselines on two well-known datasets. In the last section 4, we underline advantages and disadvantages of the architecture we proposed and we sketch future research directions.
Our model took inspiration from the model based on Long Short-Term Memory
(LSTM) network [
The proposed architecture implements a content-based recommender system able to predict a score s(u; i) which defines the probability of a like given by a user u to a specific item i. In a nutshell, our approach is based on two different modules which jointly learn a user embedding and an item embedding that are used to feed a classifier which generates the preference estimation.
Our architecture is based on a lookup table to represent users and items. Given a set of elements A, each of them can be represented as a d-dimensional vector contained in a W 2 RjAj d embedding matrix. Let ej 2 RjAj, which is all zeros except for the j-th index in which there is the value 1. For each a 2 A we assign an index j, 1 j jAj. The embedding v(ej ) associated to the element a is given by ej⊺W .
Figure 1a shows the AMAR architecture. Each user u is given to a user lookup
table (User LT ) to obtain a learnt du-dimensional user embedding v(u). Each
word w1; w2; : : : ; wm of the item description id associated to the item i is given
in input to a word lookup table (Word LT ) which generates a dw-dimensional
embedding v(wk) for each word wk, 1 k m. These word representations
v(wk) are sequentially passed through an LSTM network [
The resulting representations v(u) and v(id) are concatenated through a concatenation layer obtaining a (du + did )-dimensional feature vector given in input to a logistic regression layer to predict the score s(u; i). The list of recommendations for a given user u is generated sorting items in decreasing order by the score s(u; i) for each item i.
AMAR can be extended by adding an additional module to process supplementary features associated to each item. The AMAR extended architecture proposed in this work associates a list of genres g1; g2; : : : ; gn to each item, as shown in figure 1b. Each of them is passed in input to a genre lookup table (Genre LT ) which generates a dg-dimensional embedding v(gk) for each genre gk, 1 k n. A mean pooling layer averages the resulting representations v(gk) giving a genres embedding v(ig) which is concatenated to the user and item embeddings to evaluate the recommendation score. 3
In the experimental evaluation the performance of AMAR architectures is compared on top-n recommendation leveraging binary user preferences against two User u User LT v(u) v(w1) LSTM h(w1)
v(wm) LSTM
h(wm) Item description id w1 w2 b b b wm
Word LT
v(w2) LSTM
h(w2) Mean pooling layer
v(id) Concatenation layer Logistic regression layer (a) AMAR base
User u
Item description id
Item genres igj g1 g2 b b b gn
Genre LT v(g1) v(g2) b b b v(gn)
Mean pooling layer v(u) v(id)
v(ig) Concatenation layer
Logistic regression layer
(b) AMAR extended
state-of-the-art datasets as Movielens 1M (ML1M) 1 using 5-fold cross
validation and DBbook 2 using holdout evaluation, as in [
Table 1 shows AMAR performance against state-of-the-art algorithms (the best configurations are marked in bold) as User-to-User (U2U) and Item-to-Item (I2I) collaborative filtering, Bayesian Personalized Ranking Matrix Factorization (BPRMF), Weighted Regularized Matrix Factorization (WRMF), Sparse Linear Methods with BPR-Opt (BPRSlim) 4 and Vector Space Model with TF-IDF weighting scheme (TF-IDF) 5. Additional algorithms are proposed using pretrained word embeddings like Word2vec Google News 6 and GloVe Wikipedia 2014 + Gigaword 5 7 (# dimensions = 300). The recommendation score is generated using the cosine similarity between the item represented by averaging word embeddings of its description and the user profile represented by averaging positively rated items representations.
AMAR architectures are trained using RMSprop ( = 0:9, learning rate = 0:001) for 25 epochs by optimizing the binary cross-entropy criterion. Embedding sizes are fixed to 10. Batch sizes are 1536 for ML1M and 512 for DBbook. U2U 1 http://grouplens.org/datasets/movielens/1m/ 2 http://challenges.2014.eswc-conferences.org/index.php/RecSys 3 http://rival.recommenders.net/ 4 Implementations provided by http://www.mymedialite.net/ 5 Implementation provided by http://scikit-learn.org/ 6 https://code.google.com/archive/p/word2vec/ 7 http://nlp.stanford.edu/projects/glove/ and I2I neighborhood sizes are 30, 50, 80. BPRMF and WRMF number of factors are 10, 30, 50. The configuration with the highest average F1-measure among the chosen cut-off is reported (I2I-30, U2U-30, BPRMF-30, WRMF-50 on both datasets). 0.536 0.536 0.519 0.508 0.511 0.64 0.639 0.636 0.631 0.632 0.595 0.595 0.595 0.595 0.595
Table 1: Results of the experiments
Using a p-value of 0:05, on the DBbook dataset all the differences are statistically significant, instead on the ML1M dataset the differences between U2U and GloVe, BPRSlim and GloVe, GloVe and Word2vec are not statistically significant. All the others differences are statistically significant. 4
According to the presented results, AMAR architectures outperform all the other recommenders on the ML1M dataset. On DBbook there is not an improvement which is probably due to the very high sparsity of the dataset.
This preliminary work can be extended in different ways, such as improving training by applying early stopping and regularization techniques, by using different weight initialization strategies and by designing a proper cost function for top-n recommendation. An additional improvement could be obtained by doing hyperparameter optimization. 5
This work is supported by the IBM Faculty Award "Deep Learning to boost Cognitive Question Answering". The Titan X GPU used for this research was donated by the NVIDIA Corporation.