One crucial task in recommendation is to predict what a user will buy next given her shopping history. In this paper, we propose a novel neural network to complete this task. The model consists of an embedding layer, a hidden layer and an output layer. Firstly, the distributed representations of the user and the items bought before are obtained and used to form a feature vector by the embedding layer. Then the hidden layer transforms the feature vector to another space by a non-linear operator. Finally, the softmax operator is adopted to output the probabilities of next items. We can see that the model elegantly involves both the user's general interest and the sequential dependencies between items for prediction. Experimental results on two real datasets prove the effectiveness of our model.
Next basket recommendation (NBP) is important in applications such as online electronic business and retail shopping market. Since a user's shopping history is typically represented as a sequence of baskets, it is natural to formalize this task to a sequential prediction problem. That is, given a sequence of baskets, we want to predict what the user will buy next. In this process, both the user's general interest (what items the user likes) and the sequential dependencies between items (the in uence of items bought before to items in the next basket) are important for the prediction.
Existing approaches for this task are mainly based on
matrix/tensor factorization and markov chains [
Our model consists of three layers: embedding, hidden and output layer. The embedding layer rstly maps the user and the items in the user's shopping history to distributed representations, and then concatenates them together to obtain a feature vector. The hidden layer transforms the feature vector to another space non-linearly. Finally, the output layer gives the probabilities of next items by the softmax operator. From the above process, we can see that the two crucial factors of the user's general interest and the sequential dependencies between items are both elegantly involved in this model. Empirically, our experimental results on two 2.
Next basket recommendation can be formalized as a
sequential prediction problem, here we follow the notations
of [
Fig.1 shows our proposed model, namely NN-Rec. It is a three-layer neural network:
(1) The rst layer is the embedding layer. The inputs are the user ID and the item IDs in the user's last k baskets. We rst transform the inputs to distributed representations, where each user is represented as a vector u 2 Rdu , and each item is represented as v 2 Rdi . We obtain user matrix U 2 Rdu jUj and item matrix V 2 Rdi jIj by putting all user and item vectors together, respectively. Both U and V are learned during the training process. Then, each basket is represented as the mean of all items included in it. The output of the embedding layer is to concatenate the user's and the baskets' representations together to obtain a feature vector h1 2 Rdu+k di , which can be viewed as the representation of both the user's general interest and local context.
(2) The second layer is a non-linear hidden layer, which s = W2h2 + b2; P (ij 2 Btju; Bt 1; :::; Bt k) = esij ∑jpI=j1 esip transforms h1 to a hidden representation h2 with dimension l. Here we use tanh as the activation function, which is commonly used in neural networks. h2 is obtained as follows. where W1 2 Rl jh1j; b1 2 Rl 1 are parameters to be learned. (3) The output layer is a softmax layer, which outputs the probabilities of next items, where W2 2 RjIj l; b2 2 RjIj 1 are parameters to be learned.
For training, we use negative log-likelihood as the loss function and stochastic gradient decent with back propagation for optimization. Weight decay is also used as regularization, therefore the optimization problem becomes ∑∑
∑logP (i 2 Btju; Bt 1; :::; Bt k) u t i2Bt + U jjUjj22 + V jjV jj22 + 1jjW1jj22 + 2jjW2jj22
NN-Rec has some superiorities compared to previous stateof-the-art methods such as FPMC. Compared to FPMC which is based on tensor factorization, NN-Rec is neural network based and is more exible and powerful. Firstly, the model can easily capture longer dependencies by varying the window size k of the embedding layer, while FPMC only captures the in uence of the nearest one basket. Secondly, the embedding layer is exible and we can add other features such as user pro les and item attributes to this layer without modifying the model's framework. Finally, the hidden layer gives the power to model more complicated interactions between user and items.
We conduct experiments on two real retail datasets (Tafeng and Beiren) to evaluate the effectiveness of our model. Tafeng is a benchmark dataset released by RecSys1. While Beiren is collected by a large retail store in China, which contains users' shopping history during 2011 to 2013. We preprocess the two datasets by removing the items bought less than 10 times and the users who bought less than 10 items or 4 baskets. The detailed information of the obtained datasets are shown in Table 1. For each user, we hold out the last basket as the test set and keep other data as the training set. The last basket of all users in the training set are used for validation. The nal models are trained on the whole training set. For our model in all experiments, the dimensions of h2 (i.e., l), user vector (i.e., du) and item vector (i.e., di) are set to the same value, denoted as d. k is set to 1 and 2 since that most users only have a few baskets in the two datasets.
We recommend top c items for each user, denoted as B^ut, and use F1-measure for evaluation. c is set to 5 in Fig. 2.
F 1 = 2pr ; p = ∑u jBut \ B^utj ; r =
∑u jBut \ B^utj p + r jU j c ∑u jButj
We compare our model with several existing methods,
such as Top popular (TOP) which recommend items
according to global popularity, NMF, BPR [
Beiren NNRec(k=2) NNRec(k=1) FPMC BPR NMF TOP d d Figure 2: Experimental results on two datasets, d is the feature dimension the performances of our model are consistently better than all baselines on both datasets, whenever k = 1 or 2. Therefore, we can conclude that neural networks are suitable for this task. Furthermore, on Beiren the performance when k = 2 is consistently better than k = 1. This indicates that longer dependences between items can be captured by NNRec and are important for this task. However, on Tafeng, larger k does not show bene ts. This is because Tafeng is much smaller than Beiren, as shown in Table 1, and thus complex models are more easily to over t the training set. Therefore, we recommend to choose appropriate k according to the scale of data. 4.
In this paper, we propose to use neural networks for next basket recommendation. Our model consists three layers and can elegantly incorporate both a user's general interest and sequential dependencies between items for recommendation. Experimental results show that our model signi cantly outperforms existing approaches. To the best of our knowledge, this is the rst time to introduce neural networks to this task. We believe neural network is a more exible framework to model complicated interactions and will impact this area further. In future work we will investigate other neural networks such as recurrent neural network which can capture longer term sequential dependencies and try to incorporate other information such as time serials into the model.