Personalized ranking continues to be an important aspect of many information systems and personalization systems. Neural networks and deep learning continue to gain popularity because of their success in diferent fields of artificial intelligence such as computer vision and natural language processing. Recently, researchers began to apply deep learning to personalized ranking with success. Most personalization systems exploit historical preference data for users and items in warm-start scenario. A major challenge in personalized ranking occurs in the cold-start scenario which arises when there is little to no historical preference information. Content information is sometimes available and it can be used to alleviate the cold-start problem. We propose a solution that involves transfer learning from a deep model to a shallow model for both warm-start and cold-start personalized ranking. We corroborate our proposal with experiments on publicly available datasets in comparison with other baseline and state-of-the-art techniques.
Personalized ranking with adequate historical preference is referred to as warm-start while recommendation with inadequate historical preference is referred to as cold-start. We subsequently refer to personalized ranking as ranking except otherwise clearly stated. We propose a machine learning solution called Neural Transfer Learning for warm-start personalized ranking, otherwise referred to as NeuTraL. We then propose a cold-start version of NeuTraL referred to as NeuTraL-C. NeuTraL and The remainder of this paper is organized as follows: NeuTraL-C use neural networks and transfer learning in Section 2, we highlight related work. We provide for warm-start and cold-start item ranking respectively. pertinent background and notations for the rest of this Item cold-start personalized ranking involves ranking work in Section 3. We describe our approach in Section 5. cold-start items while user cold-start personalized rank- In Section 6, we describe our experiments and discussed ing involves ranking cold-start users. There is also the the results in section Section 6.3.3. We conclude with full cold-start entity personalized ranking problem where potential directions for future work in Section 7. both the user and item entities have no historical preference information. Although we focus on cold-start item personalized ranking in this work, we believe the concept 2. Related Work is extensible to both user cold-start and full cold-start personalized ranking problems. Entity content information is sometimes used to compensate for the lack of historical preference information by learning from content information and existing preference information. Ranking can be done for implicit or explicit feedback [1]. We focus on implicit feedback in this work due to its more prevalent nature. The contributions made in this work include: • We propose a unique approach to extracting pretrained user latent factors from a state-of-the-art (SOTA) personalization model. • The transfer of the pre-trained user latent factors to a renowned personalization model for warmstart and cold-start ranking respectively. • We provide thorough evaluation and conduct experiments comparing our proposed solutions with other SOTA and baseline techniques.
Personalized ranking techniques typically belong in one of the following categories: collaborative filtering (CF), content-based or a hybrid of the aforementioned techniques. Diferent CF techniques ranging from matrix factorization (MF) [2, 3] to k-Nearest Neighbor (kNN) [4] have seen success in personalization systems research.
In recent years, deep learning has also been successfully applied for personalization. He et al. replaced the typical dot product of user and item latent features with a deep learning model in their technique referred to as neural collaborative filtering, NCF [ 5]. NCF performs better than the vanilla MF because the non-linearity of the deep learning model captures complex interactions between users and items better. Deep representation models such as autoencoders and restricted Boltzmann machines (RBM) have been used for personalization [6, 7, 8]. These techniques have been successfully applied and demonstrated on a variety of real world data, but they are known to sufer from the cold-start problem. Content-based techniques are typically used to tackle the cold-start problem by incorporating entity attributes [9, 10]. Entity attributes are sometimes combined with CF to compensate for the weakness in CF [11, 12] for the cold-start scenario.
To alleviate the cold-start problem, some deep learning techniques have been developed with the use of content information, e.g., the deep content-based music recommendation work proposed by Oord et al. [13]. Most of the deep learning personalization systems proposed for cold start are hybrid in that they combine historical preference and content information [14, 15, 16, 17, 18, 19]. Some of the cold-start personalization systems [20] adopt active learning. However, there are situations where active feedback from users for the cold start items are unavailable.
Transfer learning has also been used in personalization systems research [21, 22].
interactions is denoted by: The set of users and items are denoted by and , respectively. A measure of preference is recorded as a positive feedback from some set or as a negative feedback recorded as 0. When explicitly provided, could be a set of values e.g., {1, 2, ..., 5}. When implicitly provided, typically = {0, 1}. The matrix of user-item
Y ∈ ({0} ∪ )||×| |, where an interaction refers to an observable action by a user e.g., the purchase of an item. User vector for user in is denoted as . Conversely, item vector for item in is denoted as . The implicit feedback for a user ∈ on an item ∈ is: = {︃1, if interacted with ;
0, otherwise. + = {set of items interacted with by user }.
− = − + +, − , and are user sets analogous to the definitions in Equations 3− m-dimensional user-attribute and n-dimensional item4. A and A represent the attribute matrices, respectively.
A ∈ R||× , U ∈ R||×
I ∈ R||× , where is the number of latent features. User and item are represented by u and i, respectively. Content data would sometimes contain only user attributes, item attributes or both. User attributes include demographic information such as age and gender, education level, etc. Social network data can also be mined for user attributes. Item attributes include physical attributes, time of production, location, etc. The task of item ranking is to estimate the relative ranking of the items for each user. We denote the predicted ranking of item for user as ˆ from an inference function :
ˆ = (u, a , i, a , ), where denotes the model parameters learned during training. Equation 9 shows ˆ is a function of the input and learned model parameters. Model parameters are typically learned via optimization such that an objective loss function is minimized or a utility function is maximized. Objective loss function minimization is expressed A
∈ R||× .
Let a be the vector of user attributes 1 . . . for user , and a be the vector of item attributes 1 . . . for item , so that is the -th item attribute value and is the -th user attribute value. = 0 when the attribute is unavailable. Sets and are represented by latent feature matrices U and I respectively where = arg min ℒ( ; Y), (10) where is learned from observation matrix Y to optimize the estimate function that predicts ˆ. Learning is usually done with machine learning techniques such as gradient descent (GD) [23] or its variants e.g., Adaptive Moment Estimation (Adam) [24] on carefully sampled user-item pairs.
We provide further background on pertinent information that will aid the understanding of NeuTraL. (1)
as: (2) (3) (4) (5) (6) (7) (8) (9) ...
...
... ... MPR 4.1. MPR: Multi-Objective Pairwise
Ranking MPR is of the pairwise ranking function family where the optimization task is with respect to the actual and predicted values for a pair of items by a user. For item ranking, the pairwise prediction function for a user , a preferred item and a less preferred item is expressed as while the actual value is ˆ(,) = ˆ − ˆ , (,) = − . ˆ(,) = ˆ − ˆ, (,) = − .
Conversely, for user ranking, the pairwise prediction function for an item preferred by user but not preferred by user is expressed as while the actual value is MPR combines item ranking and user ranking. The optimization function is expressed as: ∑︁ ∑︁ ∑︁ ∈ ∈+ ∈− ℒ(ˆ(,)) + ℒ(ˆ(,)),
(15) and the objective function ℒ is the log-sigmoid function: ˆ is estimated from a MF model learned with GD. ˆ is the dot product of the user latent vector and the item (12) latent vector . (11) (13) (14) and Assume and ℒ() = ln (), () =
1 1 + − ˆ = · = {1, 2, . . . , } = {1, 2, . . . , }.
(16) (17) (18) (19) (20)
Component of represents user ’s afinity for an item factor . Component of represents the concentration of factor in item .
· = 1 * 1 + 2 * 2 . . . , *
(21) .
Each component product * represents user ’s afinity for factor in item . We subsequently refer to this component product as latent vector product (LVP) for ease of reference. 4.2. Transfer Learning Transfer learning [25] is premised on the idea that a related pre-trained model can serve as an initializer for a main model. This initialization can be beneficial by speeding up learning and/or improving accuracy on the main task as seen in Figure 4. Transfer learning is similar to multi-task learning (MTL) with the main diference being the sequential versus simultaneous nature of the two techniques, respectively. Transfer learning has been successful in image processing [26] and natural language processing [27] among other areas of machine learning. 4.3. Auto-Encoders & Personalization (23) (24) ˆ0 = 0(, ), and 0 is a concatenation function. The nodes vector in the hidden layer are:
ˆ1 = 1(1 · ˆ0 + 1). 1 is the x ℎ weight matrix between the input and hidden layers. and ℎ are the number of nodes in the input and hidden layers respectively. 1 is the bias for the hidden layer. 1 is an activation function.
ˆ2 = 2(2 · ˆ1). 2 is the ℎ x weight matrix between the hidden and output layers. 2 is an activation function. We use sigmoid activation functions since they produced optimal results. 1, 2 and 1 are model parameters. There are also hyper-parameters such as learning rate, batch size and objective function that should be tuned during training with validation. We use the binary cross-entropy cost function.
− ˆ(,) ln − (1 − ˆ(,)) ln(1 − ). (25) and backpropagation to update the model parameters.
Auto-encoders have been successfully applied in personalization systems [7, 6]. Auto-encoders derive their name 4.4. NeuTraL Algorithm from the ability to encode input data with un-supervised learning. The utility of auto-encoders include dimension- The development of NeuTraL as depicted in fig:NeuTraL ality reduction of input while ignoring noise in the input begins with the supposition that a more representative optimally. For the purpose of personalization, entity vec- user embedding model could improve performance in tor data is passed as input with missing entries. The goal the MF for personalized ranking. A pre-trained neural is to recover the original input in the output including network model may be appropriate since we are aware the missing entries. To the best of our knowledge, the of the success of deep learning models in personalization pioneer research work in this area is AutoRec [7]. User systems. It has also been shown that neural networks are vectors or item vectors can serve as input where better at modelling complex non-linearity in user-item each vector component is the actual preference value or interactions than MF models [5]. We chose CDAE as our a missing entry. However, the authors of AutoRec stated pre-training model based on its proven improvement over that user vector inputs performed better than item vector AutoRec. User latent features in MF can be considered a inputs, and we observed the same in our experiments. form of dimensionality reduction for the user preference Perhaps this is due to the peculiar characteristics of the vector in . A close look at both CDAE and MF reveals datasets used, e.g., number of users and items, ratings that the hidden layer nodes of CDAE are analogous to per item and ratings per user. Wu et al. presented a user latent features as smaller dimension versions of the more sophisticated auto-encoder personalization tech- original user vectors in . This analogy implies we can nique, Collaborative Denoising Auto-Encoders (CDAE) use a pre-trained | | matrix of hidden layer node [6] which incorporates denoising with dropout [28] and values as the user latent feature matrix model which an extra identifier input. Dropout can be seen as a form forms the basis for our contribution. We subsequently of noise introduction [29]. refer to as the transfer matrix. In other words, we Deep learning techniques have the advantage of being transfer user vector from as the latent vector for able to model linear and non-linear complex interactions user u. We leave out the algorithm for NeuTraL since it between users and items. Auto-encoders for personal- is essentially the same as the MPR algorithm with the ization are depicted in Figure 1. We denote the nodes in use of the pre-trained user embedding from CDAE. the input layer as ˆ0, hidden layer as ˆ1 and the output layer as ˆ2 where ...
... k ...
k t
matrix ... t
tributes ... ... ...
We provide further background on pertinent information that will aid the understanding of NeuTraL-C as depicted in fig:NeuTraL-C. 5.1. Item Attribute-to-Feature Mappings Cold-start items have little to no historical preference information to exploit for personalized ranking. Hence recommending cold-start items pose a diferent challenge.
However, both warm-start and cold-start items have item attributes that can be exploited for recommendations.
An item Attribute-to-Feature Mapping (ATM) as a framework capable of providing item latent features from item attributes i.e., a function that accepts item attributes as input and produces item latent features as output. The output can then be used in conjunction with user latent features for prediction. We consider the ATM technique presented by Gantner et al [12] referred to as ATM-BPR in this work. ATM-MPR is an extension of the ATM-BPR technique for cold-start personalization. 5.1.1. ATM-MPR ATM-MPR adds cold-start capability to MPR by learning a shallow linear model of latent features and attributes.
The main diferences between MPR and ATM-MPR is the derivation of the item latent vector where and ℳ is a mapping function.
= ℳ( ), where is a mapper matrix to be learned similar to how and are learned in MPR with GD. MPR optimizes the NeuTraL-C optimization criterion which is the same as neutral-opt.
However, the respective prediction functions for user ranking and item ranking in NeuTraL-C are diferent.
We subsequently describe the item ranking prediction function but the user ranking prediction function is analogous. The item ranking prediction function is expressed as: Algorithm 1 NeuTraL-C(, , ) 1: Output: Optimized matrices U and M 2: initialize U with the extracted hidden layer matrix C
from CDAE 3: initialize , and M 4: repeat 5: draw , , from , +, − uniformly 6: ← − * -C-
← − * -C- wrt and 7: draw , , from , +, − uniformly 8: ← − * -C- wrt and ← − * -C- ← − * -C- 9: until convergence or maximum number of iterations 10: return U, M 6.1. Experimental Repeatability ˆ(,) = ( · · ) − ( · · ). (28) Experiment Artifacts (software, datasets, etc.) for this work are available on demand. These artifacts will be With transfer learning, the prediction function becomes: made publicly available with publication. All of the techniques use GD and/or Adam for training as is the case in ˆ(,) = ( · · ) − ( · · ). (29) NeuTraL where we use Adam for pretraining CDAE but use GD for actual training in the ATM-BPR framework. ˆ(,) = ( − ). (30) The benchmarks will converge diferently during training based on hyperparameters but 1 factor that afects Hence, is updated in GD with the following expres- the space and time requirements during each epoch is sion: the size of model parameters. Avoidance of bias forms the basis for model design and other hyperparameter se = + ( -C- ) , (31) lections throughout our experiments. We use one hidden layer in the deep models. We use 100 factors in the MF models. We also have the number of nodes in the deep = + ︃( ℒ(ˆˆ((,,))) · ˆ(,) − · )︃ , lteeacrtnuirnegfomrotdheel daemeopulneatrtnoi1n0g0m.Wodeeulss.eWdethuesteodwleeraranrcinhgi(32) rates between 0.00001 − 0.01 and batch sizes of 10000. and is a regularization hyper-parameter. We tuned model hyperparameters and stopped training early with validation. 5.2. NeuTraL-C Algorithm The NeuTraL-C algorithm is listed in alg-neutral-c • how does NeuTraL compare with other SOTA
warm-start item personalization systems. • how does NeuTraL-C compare with other SOTA cold-start item personalization systems.
6.2. Evaluation metrics
We begin by describing our experiment setup. We subsequently describe our experiments on warm-start personalized ranking followed by cold-start. 6.3. Experiments for warm-start ranking 6.3.1. Datasets We performed experiments on four publicly available datasets. A summary of these datasets is provided in Table 1. The datasets contain explicit ratings for users on items but we convert the ratings to implicit feedback by treating ratings greater than 0 as positive feedback.
Our focus in this work is implicit feedback but we believe NeuTraL is applicable to explicit feedback.
• Movielens 1M: Movielens dataset of diferent datasets [30] are made publicly available by the GroupLens Research lab at the University of Minnesota. We use the Movielens 1M dataset. The data is extracted from the Movielens website which is a free website that provides personalized movie recommendation to users. • Eachmovies dataset: This dataset [31] is made available by the Digital Equipment Corporation (DEC) Systems Research Center at Compaq. The research center ran a CF service for experimental purposes and made the data available for research. • Goodreads dataset: This dataset [32] was collected from goodreads.com, a book social network and recommendation website. • Pinterest Dataset: This is a dataset of implicit feedback representing whether a user pinned an image on their board on the pinterest platform at https://www.pinterest.com. 6.3.2. Benchmarks We compare our NeuTraL technique with 3 SOTA coldstart personalization systems and a baseline item popularity (IPop) technique. IPop recommends items based on popularity. The benchmarks will converge diferently during training based on hyperparameters but 1 factor that afects the space and time requirements during each epoch is the size of model parameters. We select model parameters to avoid bias throughout our experiments.
The SOTA benchmarks used are described below: • BPR: we described BPR in bpr.
MRR NDCG AUC
MRR NDCG AUC
MRR NDCG AUC We record the best average results observed during experiments for each dataset and depict them in movielens-table,eachmovie-table. NeuTraL significantly out-performs the other techniques based on a Wilcoxon signed-rank test with a -value < 0.01. The winning algorithm per metric is emboldened in each row of all tables. We assume a margin of error of 0.005, hence the winning algorithm has to be greater than the next winner by at least a margin of 0.005. All techniques are emboldened in the case of a tie on a metric. Techniques within the margin of error of the highest score are also emboldened. 6.4. Experiments for cold-start ranking 6.5. Datasets We performed experiments on 3 of the 4 publicly available datasets used for warm-start experiments in section warm-start-datasets. We used the datasets with item attributes, hence their suitability for our experiments. A summary of these datasets is provided in warmdatasetstable. The 3 datasets used for cold-start personalization experiments are highlighted below: • Movielens 1M: Item attributes in the dataset include release year and genre. The genre attribute is one-hot encoded into 18 dimensions because we have 18 possible genres. The year is an additional dimension. • Eachmovies dataset: The items/movies in this dataset are a subset of the items in the Movielens dataset, hence we are able to us the same attribute feature engineering as described for Movielens. • Goodreads dataset: We use the genres as book attributes for cold-start personalization. The genre attribute is one-hot encoded into 10 dimensions because we have 18 possible genres.
6.5.1. Benchmarks We compare our NeuTraL technique with 4 state-of-theart cold-start personalization systems. NeuTraL-C, DropouNet and ATM-BPR require pre-training. The benchmarks used are described below: • Multi-layer perceptron (MLP): The MLP baseline used here predicts output from interactions between user embedding and item attributes with deep learning. The first hidden layer is the input combination layer that combines user embedding input and item attributes. The combination model is the piece-wise product since this has been demonstrated to outperform concatenation or a dot product [34]. The dot product also doesn’t allow us assign diferent weights to the combined nodes. The output from this combination layer are propagated through extra hidden layers. More hidden layers can be added as needed before the ifnal output. • ATM-BPR The ATM-BPR technique used a baseline here is described in atm-bpr except the pretrained user embedding is extracted from BPR instead of an CDAE recommender which is used in NeuTraL-C. • DropoutNet: Addressing Cold Start in Recommender Systems DropoutNet [22] is a state-of-theart deep learning based personalization system.
DropoutNet is analogous to NeuTraL and ATMBPR. DropoutNet adopts a diferent transfer learning procedure compared to NeuTraL. DropoutNet transfers a pre-trained shallow model to a deep model while NeuTraL transfers a pre-trained deep model to a shallow model. We use the MLP model described here as the deep learning model. DropoutNet allows the use of diferent pretrained models but we use pre-trained user latent features from CDAE similar to NeuTraL-C i.e. the DropoutNet implementation used here is a combination of the extracted user latent factors from CDAE and MLP. Although DropoutNet is primarily a cold start recommender but it is expected to perform relatively well on warm start recommendations with the appropriate dropout rate. We use a maximum input dropout rate of 1.00 for our experiments with DropoutNet to maximize performance on cold-start because that is the focus of this research work. DropoutNet also allows inference transform but we do not apply it in our experiments because we do not consider the case of incremental item preference data collection as described in their work. We refer to DropoutNet as D-Net to conserve space in the results tables. • W&D: Wide & Deep Learning for Recommender
Systems W&D [19] combines generalization and
MRR NDCG
AUC
MRR NDCG
AUC memorization capabilities of recommender sys- in each row of all tables. We assume a margin of error tems for more robust personalization. They used of 0.005, hence the winning algorithm has to be greater deep learning for its demonstrated superior gener- than the next winner by at least a margin of 0.005. All alization capability. However, deep learning tends techniques are emboldened in the case of a tie on a metto over-generalize when the input is too sparse ric. Techniques within the margin of error of the highest and high-rank. On the other hand, generalized score are also emboldened. linear models are highly capable of memorization of feature interactions through cross product fea- 6.6. Discussion ture transformations. Hence, the combination of a deep learning and a cross product model (wide) in W&D for personalization. 6.5.2. Evaluation metrics for cold-start We measured how well a recommender system is able to rank a preferred cold-start item relative to other items.
The evaluation is similar to the evaluation for warm-start items. The main diference is the absence of test items in the training dataset for cold-start personalized ranking. 6.5.3. Results We record the best results observed during experiments for each dataset and depict them in movielens-table-coldstart,eachmovies-table-cold-start. NeuTraL-C performs best overall and we subsequently discuss the results further. The winning algorithm per metric is emboldened We begin our discussion with the results of the warmstart experiments. We stated that NeuTraL performed best overall because of its highest number of wins which corresponds to the number of times a technique has the highest score per dataset. We also validated this observation with a significance test. IPop has the worst performance overall. This is not surprising since it is merely a baseline technique that ranks items based on popularity.
The ranking produced by IPop is not personalized as it does not take personal attributes, context or historical preference into account. We expect a decent personalized ranking technique to out-perform IPop. This is the case as least performing personalized ranking technique is BPR but it ourperforms IPop. NCF performs better than BPR. This was already demonstrated by the creators of NCF in their research work [5]. NCF combines both deep learning (MLP) and piecewise product of interactions between user and item embeddings in a generalized
MRR NDCG AUC matrix factorization (GMF). BPR uses a dot product of instance, the transferred user embedding is propagated user and item embeddings to represent the interactions. through hidden layers before combination with the item Dot product assigns equal weights to the LVPs as de- attributes. The output of the hidden layers is a tainted scribed in dot-product while the GMF component of NCF version of the user embedding. The mapping learned by learns diferent weights for the LVPs with a neural net- DropoutNet is between this tainted version and the item work. The MLP component of NCF also learns diferent attributes. We believe this is the reason for a poorer perweights for user and item embedding combinations. This formance compared to ATM-BPR and NeuTraL-C. It is not results in more complex representation of interactions too surprising that MLP performed less than DropoutNet between users and items and better performance. MPR since it is DropoutNet without transfer learning. Once out-performs NCF. The MTL nature of MPR gives it an again, this shows the efectivenes of transfer learning. advantage. NeuTraL’s superior performance butrresses WD performed the least of all cold-start personalization the efectiveness of transfer learning since it is essentially systems. It does not use transfer learning and we beMPR combined with transfer learning but it outperforms lieve the complexity of deep learning in WD deteriorated MPR. We surmise that transfer learning improved the performance due to overfitting. performance of NeuTraL. We also believe that the type of A common theme throughout or experiments is the pre-trained model that is transferred is significant. Our benefit of our neural transfer learning approach. We experiment here reveals that the extraction mechanism believe that the transferred user embedding is more repfrom an autoencoder based model like CDAE is efective. resentative of the users as latent factors compared to the
We subsequently discuss the results of our experiments user embedding in the other models. We show a chart on cold-start personalization. We stated that NeuTraL-C of loss minimization in NeuTraL with and without transperformed best overall because of its highest number of fer learning in 3 on the Movielens data. 3 shows the wins which corresponds to the number of times a tech- speed-up achieved with transfer learning in the form of nique has the highest score per dataset. We also validated lower initial loss. 3 also shows the overall lower loss with this observation with a significance test. ATM-BPR is training. We know that ATM-BPR and DropoutNet adopt the next best performing technique. Both ATM-BPR and transfer learning as well but are outperformed by NeuNeuTraL-C adopt transfer learning. However, NeuTraL- TraL. As stated earlier in section:cdae, dropout is a vital C uses a diferent pre-trained model. NeuTraL-C uses a component of CDAE, hence we investigated the efect pre-trained model extracted from CDAE as described in of dropout when pre-training on the final results. The section:NeuTraL while ATM-BPR uses pre-trained user results show that dropout slightly enhances the efect of embedding from BPR. This shows that it is not enough the transferred user embedding in NeuTraL. to just apply transfer learning but the meticulousness of implementation is as important. The type of pre-trained model is pertinent in such design. NeuTraL-C and ATM- 7. Conclusion BPR also difer in how they learn the "mapping function". NeuTraL-C uses MPR while ATM-BPR uses BPR. We presented a novel personalization system based on DropoutNet performs next best to ATM-BPR. Dropout- transfer learning from a state-of-the-art deep personalizaNet also uses transfer learning. We used user embed- tion system to a linear cold-start personalization model. ding from CDAE in DropoutNet. However, it uses deep This system is applicable to warm-start and cold-start learning to learn the interaction between the transferred items and users. The results of our experiments show the embedding and item attributes. The complex nature of efectiveness of our proposed method and we discussed DropoutNet deteriorated performance somewhat. For the results. Although the results are promising, there is room for future work and improvements. Potential future research work include the extension of our techniques to user cold-start, full cold-start and warm-start ranking. Other potential future work includes investigation of additional attributes and optimum fusion strategy of those attributes. We believe experimentation with more datasets and context attributes such as time and location would also be worthwhile. Neural Information Processing Systems - Volume knowledge, in: 2013 IEEE International Conference 2, NIPS’13, Curran Associates Inc., USA, 2013, pp. on Multimedia and Expo (ICME), 2013, pp. 1–6. 2643–2651. URL: http://dl.acm.org/citation.cfm?id= [22] M. Volkovs, G. Yu, T. Poutanen, Dropoutnet: Ad2999792.2999907. dressing cold start in recommender systems, in: [14] P. Covington, J. Adams, E. Sargin, Deep neural net- I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, works for youtube recommendations, in: Proceed- R. Fergus, S. Vishwanathan, R. Garnett (Eds.), Adings of the 10th ACM Conference on Recommender vances in Neural Information Processing Systems Systems, New York, NY, USA, 2016. 30, Curran Associates, Inc., 2017, pp. 4957–4966. [15] T. T. Nguyen, H. W. Lauw, Collaborative topic re- [23] C. Burges, T. Shaked, E. Renshaw, A. Lazier, gression with denoising autoencoder for content M. Deeds, N. Hamilton, G. Hullender, Learning to and community co-representation, in: Proceed- rank using gradient descent, in: Proceedings of the ings of the 2017 ACM on Conference on Infor- 22Nd International Conference on Machine Learnmation and Knowledge Management, CIKM ’17, ing, ICML ’05, ACM, New York, NY, USA, 2005, ACM, New York, NY, USA, 2017, pp. 2231–2234. pp. 89–96. URL: http://doi.acm.org/10.1145/1102351. URL: http://doi.acm.org/10.1145/3132847.3133128. 1102363. doi:10.1145/1102351.1102363. doi:10.1145/3132847.3133128. [24] D. P. Kingma, J. Ba, Adam: A method for [16] H. Wang, N. Wang, D.-Y. Yeung, Collaborative deep stochastic optimization., CoRR abs/1412.6980 learning for recommender systems, in: Proceedings (2014). URL: http://dblp.uni-trier.de/db/journals/ of the 21th ACM SIGKDD International Conference corr/corr1412.html#KingmaB14. on Knowledge Discovery and Data Mining, KDD [25] L. Torrey, J. Shavlik, Transfer learning, 2009. ’15, ACM, New York, NY, USA, 2015, pp. 1235–1244. [26] A. Quattoni, Transfer learning algorithms for image URL: http://doi.acm.org/10.1145/2783258.2783273. classification, Ph.D. thesis, Citeseer, 2009. doi:10.1145/2783258.2783273. [27] N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, [17] G. Sottocornola, F. Stella, M. Zanker, F. Canonaco, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, Towards a deep learning model for hybrid rec- S. Gelly, Parameter-eficient transfer learning for ommendation, in: Proceedings of the Interna- nlp, arXiv preprint arXiv:1902.00751 (2019). tional Conference on Web Intelligence, WI ’17, [28] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, ACM, New York, NY, USA, 2017, pp. 1260–1264. R. Salakhutdinov, Dropout: A simple way to preURL: http://doi.acm.org/10.1145/3106426.3110321. vent neural networks from overfitting, J. Mach. doi:10.1145/3106426.3110321. Learn. Res. 15 (2014) 1929–1958. URL: http://dl.acm. [18] W. Niu, J. Caverlee, H. Lu, Neural personalized org/citation.cfm?id=2627435.2670313. ranking for image recommendation, in: Proceed- [29] C. M. Bishop, Training with noise is equivalent ings of the Eleventh ACM International Confer- to tikhonov regularization, Neural computation 7 ence on Web Search and Data Mining, WSDM ’18, (1995) 108–116.
Association for Computing Machinery, New York, [30] F. M. Harper, J. A. Konstan, The movielens datasets: NY, USA, 2018, p. 423–431. URL: https://doi.org/ History and context, ACM Trans. Interact. Intell. 10.1145/3159652.3159728. doi:10.1145/3159652. Syst. 5 (2015) 19:1–19:19. URL: http://doi.acm.org/ 3159728. 10.1145/2827872. doi:10.1145/2827872. [19] H.-T. Cheng, L. Koc, J. Harmsen, T. Shaked, T. Chan- [31] P. McJones, Eachmovie Collaborative Filterdra, H. Aradhye, G. Anderson, G. Corrado, W. Chai, ing Dataset, DEC Systems Research CenM. Ispir, et al., Wide deep learning for recom- ter,http://www.research.compaq.com/src/eachmovie/, mender systems, in: Proceedings of the 1st Work- 1997. shop on Deep Learning for Recommender Systems, [32] M. Wan, J. J. McAuley, Item recommendation on DLRS 2016, Association for Computing Machin- monotonic behavior chains, in: S. Pera, M. D. ery, New York, NY, USA, 2016, p. 7–10. URL: https: Ekstrand, X. Amatriain, J. O’Donovan (Eds.), Pro//doi.org/10.1145/2988450.2988454. doi:10.1145/ ceedings of the 12th ACM Conference on Rec2988450.2988454. ommender Systems, RecSys 2018, Vancouver, BC, [20] Y. Zhu, J. Lin, S. He, B. Wang, Z. Guan, H. Liu, Canada, October 2-7, 2018, ACM, 2018, pp. 86– D. Cai, Addressing the item cold-start problem by 94. URL: https://doi.org/10.1145/3240323.3240369. attribute-driven active learning, IEEE Transactions doi:10.1145/3240323.3240369. on Knowledge and Data Engineering 32 (2020) 631– [33] R. Otunba, R. A. Rufai, J. Lin, Mpr: Multi-objective 644. pairwise ranking, in: Proceedings of the Eleventh [21] Ming Yan, Jitao Sang, Tao Mei, Changsheng Xu, ACM Conference on Recommender Systems, RecFriend transfer: Cold-start friend recommenda- Sys ’17, Association for Computing Machinery, tion with cross-platform transfer learning of social New York, NY, USA, 2017, p. 170–178. URL: https: //doi.org/10.1145/3109859.3109903. doi:10.1145/ 3109859.3109903. [34] R. Otunba, R. A. Rufai, J. Lin, Deep stacked ensemble recommender, in: Proceedings of the 31st International Conference on Scientific and Statistical Database Management, SSDBM ’19, Association for Computing Machinery, New York, NY, USA, 2019, p. 197–201. URL: https://doi.org/10.1145/3335783. 3335809. doi:10.1145/3335783.3335809.