=Paper=
{{Paper
|id=Vol-1922/paper7
|storemode=property
|title=Accurate and Diverse Recommendation based on Users’ Tendencies toward Temporal Item Popularity
|pdfUrl=https://ceur-ws.org/Vol-1922/paper7.pdf
|volume=Vol-1922
|authors=Koki Nagatani,Masahiro Sato
|dblpUrl=https://dblp.org/rec/conf/recsys/NagataniS17
}}
==Accurate and Diverse Recommendation based on Users’ Tendencies toward Temporal Item Popularity==
https://ceur-ws.org/Vol-1922/paper7.pdf
Accurate and Diverse Recommendation based on
Users’ Tendencies toward Temporal Item Popularity
Koki Nagatani Masahiro Sato
Fuji Xerox Co., Ltd. Fuji Xerox Co., Ltd.
6-1 Minatomirai, Nishi-ku, Yokohama, Japan 6-1 Minatomirai, Nishi-ku, Yokohama, Japan
nagatani.koki@fujixerox.co.jp sato.masahiro@fujixerox.co.jp
ABSTRACT To produce personalized recommendations, collaborative filtering
Popularity bias is a phenomenon associated with collaborative fil- (CF) is a widely used approach. The CF approach produces items
tering algorithms, in which popular items tend to be recommended for a target user using data compiled from observations of users
over unpopular items. As the appropriate level of item popular- with similar preferences as the target user [9]. The CF approach
ity differs depending on individual users, a user-level modification is categorized into two types: neighborhood-based CF [12, 13] and
approach can produce diverse recommendations while improving model-based CF [5, 11]. The standard approach of model-based CF
the recommendation accuracy. However, there are two issues with is a matrix factorization (MF)-based approach, which character-
conventional user-level approaches. First, these approaches do not izes both items and users by vectors of latent factors inferred from
isolate users’ preferences from their tendencies toward item pop- user feedback [5, 11]. In most cases, model-based CF is superior to
ularity clearly. Second, they do not consider temporal item pop- neighborhood-based CF in terms of accuracy.
ularity, although item popularity changes dynamically over time In the CF approach, it has been noted that popular items tend
in reality. In this paper, we propose a novel approach to counter- to be recommended more often [15, 21]. This is known as pop-
act the popularity bias, namely, matrix factorization based collab- ularity bias and various solutions have been proposed to tackle
orative filtering incorporating individual users’ tendencies toward this problem [3, 4, 7, 10, 21]. These solutions are classified into
item popularity. Our model clearly isolates users’ preferences from two types according to the level of modification: global-level and
their tendencies toward popularity. In addition, we consider the user-level. Global-level solutions modify their recommendations
temporal item popularity and incorporate it into our model. Ex- for all users uniformly by avoiding recommending popular items
perimental results using a real-world dataset show that our model [3, 7, 21]. In reality, however, the appropriate level of modification
improve both accuracy and diversity compared with a baseline al- differs depending on the user: some users are likely to select popu-
gorithm in both static and time-varying models. Moreover, our lar items, while others tend to seek new or niche items. Therefore,
model outperforms conventional approaches in terms of accuracy user-level modification approaches, in which the degree of modifi-
with the same diversity level. Furthermore, we show that our pro- cation varies according to individual users’ popularity tendencies,
posed model recommends items by capturing users’ tendencies to- have been proposed [4, 10].
ward item popularity: it recommends popular items for the user However, there are two issues in conventional user-level ap-
who likes popular items, while recommending unpopular items for proaches. First, these approaches do not isolate users’ preferences
those who don’t like popular items. from their popularity tendencies clearly. Second, although item
popularity changes dynamically over time in reality, these approaches
CCS CONCEPTS do not consider temporal item popularity. In general, incorporat-
ing temporal item popularity into models improves the recom-
• Information systems → Personalization; Recommender sys-
mendation accuracy. Moreover, to counteract popularity bias, es-
tems;
pecially in user-level solutions, incorporating temporal item pop-
ularity is important because the reasons of users’ behaviors are
KEYWORDS considered different depending on their purchase time even if they
popularity bias, temporal information, personalized recommenda- purchase same items. To the best of our knowledge, however, there
tion is no approach considering temporal item popularity in the field of
counteraction against popularity bias.
In this paper, we propose a novel approach to tackle the popular-
1 INTRODUCTION
ity bias, namely, MF-based CF incorporating item popularity ori-
Recommender systems help users to access the specific informa- entation of individual users. Our model isolates users’ preference
tion that they seek from a huge amount of data. Accurate recom- from their tendencies toward item popularity clearly. We also con-
mendations lead to an increase in customers’ purchases or con- sider temporal item popularity and incorporate it into our model.
sumption; hence, there is a need for more efficient recommender To verify the efficacy of the proposed model, we conducted exper-
systems that produce personalized content for individual users. iments using a real-world dataset. The experimental results show
that our model improves both accuracy and diversity compared
TempRRS ’17, August 2017, Como, Italy with a baseline algorithm in both static and time-changing mod-
Copyright © 2017 for this paper by its authors. Copying permitted for private and els. Moreover, our model outperforms conventional approaches in
academic purposes.
TempRRS ’17, August 2017, Como, Italy Koki Nagatani and Masahiro Sato
terms of accuracy with the same diversity level. We also demon- the time on Netflix data. Since then, several models that consider
strate that our proposed model recommends items by capturing temporal dynamics using MF [1, 20] or deep learning methods
users’ tendencies toward item popularity: it recommends popu- [16, 19] have been proposed. In user-level approaches to popular-
lar items to users who like popular items, and unpopular items to ity bias, temporal item popularity needs to be considered to cap-
those who do not like popular items. ture individual users’ tendencies toward item popularity. This is
We summarize the main contribution of this paper as follows: because the reasons for purchasing items in case of users having
• Our model isolates users’ preferences from their tendencies multiple interactions with the same items may be different depend-
toward item popularity clearly. ing on the interaction time: some users purchase items because
• We consider temporal item popularity in the field of coun- the items are popular, and some users purchase items because the
teraction against popularity bias. items match the users’ preferences. To our knowledge, however,
• We conduct experiments using a real-world dataset to verify there is no approach that considers temporal aspects in the field of
the efficacy of the proposed model. popularity bias.
2 RELATED WORK 3 OUR MODEL
2.1 Popularity Bias In this section, we present our MF-based model that incorporates
Popularity bias is a phenomenon of existing recommendation al- individual users’ tendencies toward item popularity. We focus on
gorithms in which popular items tend to be recommended over situations where personalized top-N recommendations are pro-
unpopular items. To tackle this problem, several approaches have duced based on users’ implicit feedback (e.g. views, clicks, pur-
been proposed [3, 4, 7, 10, 21]. These approaches are classified into chases, etc.).
two types according to the level of modification: global-level and
user-level. 3.1 Modeling Individual Users’ Tendencies
Global-level approaches modify their recommendations for all toward Temporal Item Popularity
users uniformly [3, 7, 21]. Most methods avoid recommending pop- In MF, both items and users are characterized by vectors of latent
ular items by weighting according to item popularity. In global- factors derived from explicit feedback (e.g. ratings) as well as im-
level approaches, the evaluation metrics such as diversity and nov- plicit feedback. The basic model of MF with item bias is formulated
elty improve at the cost of a decline in accuracy. Generally, the ap- as follows:
propriate level of modification differs depending on the user: some
users are likely to select popular items, some tend to seek new or x̂ui = bi0 + fuT fi , (1)
niche items, and some select an item irrespective of its popularity.
However, global-level approaches do not consider such individual where x̂ui is the prediction score of preference of user u toward
differences. item i, bi0 is an item-specific bias which represents item popular-
User-level approaches consider these differences and then mod- ity, and fu and fi are k-dimensional vectors of latent factors of
ify their recommendations depending on the individual user’s ten- user u and item i, respectively. The inner product fuT fi achieves a
dencies toward item popularity. Therefore, user-level approaches high value when both user and item vectors are similar. Further-
possibly improve both diversity and accuracy simultaneously. The more, item bias bi increases when an item is popular. The predic-
conventional user-level approaches proposed in [4, 10] attempt to tion score is determined by their aggregation.
re-rank recommendation lists by post sampling based on users’ If item bias bi values are extremely high, the item is recom-
past behavior in terms of popularity. However, users’ preferences mended regardless of whether users like it or not. Hence, recom-
and their tendencies toward item popularity might be mixed in mendation systems tend to recommend these items, which leads
these approaches for two reasons. First, before reranking, the rec- to popularity biased recommendation. A simple solution for this
ommendation lists are created by existing CF models. During the problem is to penalize items according to the item popularity. How-
creation process, these models mix users’ preference and item pop- ever, preference toward popular or unpopular items varies for each
ularity. Second, popularity tendency distributions are created based user. Considering this, the solution is not suitable for users who
on users’ past actions. As users’ past actions are mainly derived like popular items. Therefore, the penalization of popularity needs
from the users’ preferences and items’ popularity, these aspects are to be changed depending on the users’ popularity tendencies.
also included when creating the distribution. Therefore, these ap- Moreover, the users’ popularity tendencies should be consid-
proaches do not isolate user preferences from their popularity ten- ered along with the items’ temporal aspects for two reasons. Firstly,
dencies clearly. Our solution overcomes the above issue by mod- item popularity changes dynamically over time in the real world
eling users’ popularity tendencies directly, as described in Section for various reasons [17]. Secondly, the reasons for purchasing items
3. in case of users having multiple interactions with the same items
may be different depending on the interaction time.
2.2 CF with Temporal Aspects Therefore, we develop a model to incorporate both users’ popu-
Incorporating temporal aspects into CF has been investigated, par- larity tendencies and items’ temporal popularity, which is formu-
ticularly for developing accurate recommendation algorithms. For lated as follows:
example, [6] proposed a matrix factorization model that considered
x̂ui = (bi0 + bi (t))(1 + дu ) + fuT fi , (2)
temporal dynamics and achieved state-of-the-art performance at
Accurate and Diverse Recommendation based on
Users’ Tendencies toward Temporal Item Popularity TempRRS ’17, August 2017, Como, Italy
where дu is the user-specific parameter of popularity tendency and period. The top-N prediction precision is defined as:
bi (t) is the time-varying item bias at the period of time t. The pa-
1 ∑ 1 ∑ |Iu
pred
rameters, bi0 , bi (t), дu , fu , and fi , are learned by optimization. (t) ∩ Iutrue (t)|
Precision@N = ,
The дu value works as the balancing parameter between the |U | |Tu | N
u ∈U t ∈Tu
item popularity and preference toward the item. When the дu value
of a user u is greater than zero, the user prefers popular items to pred
where |Iu (t)| = N , U is the set of users in the testing set and
unpopular items. High дu values indicated that the user may sim- Tu is the set of the period of time when interactions of user u are
ply prefer popular items without regard to his/her item preference. observed. Similarly, the top-N prediction recall is defined as:
Conversely, when it is less than minus one, the user prefers unpop-
1 ∑ 1 ∑ |Iu
ular items to popular items. pred
(t) ∩ Iutrue (t)|
As mentioned in Section 2.1, users’ preferences and their ten- Recall@N = .
|U | |Tu | |Iutrue (t)|
dencies toward item popularity are mixed in conventional user- u ∈U t ∈Tu
level approaches. In contrast, our model resolves the confusion by The top-N item coverage applies to all the output that a recom-
modeling as in Eq. 2: the first term represents item popularity and mender system produces for a set of users. This metric is also called
users’ popularity tendencies, and the second term represents item the top-N aggregate diversity. In our experiment, this metric is de-
feature and users’ preference. Therefore, our model captures these fined as:
features separately. ∑ ∪
t ∈T | u ∈Ut Ru |
Coverage@N = ,
3.2 Model Learning |T |
Our model formulated in Eq. 2 can be learned by applying exist- where Ut is a set of users whose interactions are observed at a
ing optimization methods, such as point-wise and pair-wise op- period of time t and Ru is the recommendation lists for user u, and
timization. For example, for point-wise optimization, root mean the length of the lists is N .
square error (RMSE), which is used in Biased-MF [5], and alter-
nating least squares, which is used in weighted regularize matrix 4.3 Comparison of Methods
factorization [2] can be applied to our model. For pair-wise op-
To examine the performance of our proposed methods, we com-
timization, area under the curve (AUC) in Bayesian personalized
pared them with conventional approaches. For the optimization
ranking [11], mean reciprocal rank used in collaborative less-is-
of our methods and base models of conventional approaches, we
more filtering (CLiMF) [14], and weighted approximately ranked
selected the Bayesian personalized ranking (BPR) procedure [11],
pairwise loss proposed in [18] can be applied to our models.
which is one of the state-of-the-art methods for personalized item
recommendation. The model of BPR matrix factorization (BPRMF)
4 EXPERIMENTS is formulated in Eq. 1. For the baseline of the conventional methods
In this section, we conduct experiments using a real-world dataset that consider temporal aspects, we extend BPRMF incorporating
to verify the efficacy of the proposed model. temporal item popularity, which is called BPRMF(t).
Personal Popularity Tendency Matching (PPTM) [10] is a greedy
4.1 Dataset re-ranking method that considers an individual’s personal popu-
We used the Amazon.com Movies and TVs dataset [8] in our exper- larity tendency (PPT). It balances novelty and user preference by
iment. We utilized a subset from 2013, defined the period of time t matching the PPT of a recommendation to that of the users mea-
as monthly, and binarized the data treating reviewed items as rel- sured by earth movers distance (EMD), which is a distance metrics
evant and non-reviewed items as irrelevant. Due to the sparsity of between two distributions.
the dataset, we preprocessed it by retaining the top 10, 000 items Personalized Ranking Adaptation (PRA) [4] is a versatile greedy
and discarding data of users having less than 10 interactions. Af- re-ranking method that considers an individual user tendency suit-
ter the preprocessing, the total number of users was 4, 997 and the able for multiple optimization goals. In our experiments, the opti-
dataset contained 90, 341 interactions for 9, 221 items. mization target is set to EMD.
BPRMF(t)-pop is the method proposed by this paper in Eq. 2.
BPRMF-pop is the model that removes temporal item popularity
4.2 Evaluation Metrics from Eq. 2.
In our experiments, we performed five-fold cross validation and To model PPT, the discrete distribution of the binned popularity
aggregated the results. First, we randomly selected 80% of observed values of the items is required. In our experiments, we defined the
feedback as a training set to train models, and the remaining 20% item popularity of the recommendations as the number of item oc-
as the testing set for the trained models. To measure the perfor- currences in the top-N recommendation lists for the active users.
mance, we used three evaluation metrics: the top-N prediction pre- We used a log-scaled popularity histogram for discrete distribu-
cision (Precision@N), the top-N prediction recall (Recall@N), and tion. The parameters of all models were tuned so as to maximize
pred
the top-N item coverage (Coverage@N). We set Iu (t) as the pre- the accuracy metrics. In the case of conventional approaches, it is
dicted items of user u over a certain period of time t, and Iutrue (t) known that a higher coverage setting reduces accuracy. Hence, we
as the true list in the testing set. Prediction is performed for each selected the parameter value for which the coverage score became
period of time t, and each user’s scores are aggregated over each close to that of our model.
TempRRS ’17, August 2017, Como, Italy Koki Nagatani and Masahiro Sato
Table 1: Precision@10, Recall@10, and Coverage@10 scores Table 2: The bi (t) and preference score of Top-5 Recommen-
on Amazon.com Movies and TVs datasets (#factors = 300). dation of BPRMF(t)-pop for a user at time t. Actual user be-
havior (relevant recommendation) is bolded. Ranking is cal-
Methods Precision@10 Recall@10 Coverage@10
culated based on the training set. "-" means the item is not
BPRMF 0.01349 0.07492 0.6464 in the training set.
+PPTM (c = 0.1) 0.01348 0.07487 0.6509
(a) Item popularity orientation score дu = 0.69, a user likes popular items.
+PRA (X u =5) 0.01274 0.07012 0.7029
BPRMF-pop 0.01359 0.07550 0.6528 TopN t =3 t = 12
items
BPRMF(t) 0.01521 0.09314 0.4939 bi (t ) (#rank) Pref. score bi (t ) (#rank) Pref. score
+PPTM (c = 1) 0.01504 0.09047 0.5634
1 4.17 (1) 0.06 4.63 (4) 0.041
+PRA (X u =5) 0.01308 0.07764 0.5605
2 3.95 (3) 0.02 4.33 (1) -0.017
BPRMF(t)-pop 0.01603 0.09569 0.5749
3 3.73 (4) 0.02 4.14 (3) -0.017
4 3.62 (14) 0.10 4.10 (6) -0.028
5 3.67 (8) -0.02 4.04 (7) 0.012
(b) Item popularity orientation score дu = −1.19,
a user selects items which match user’s preferences.
TopN t =2 t =3
items
bi (t ) (#rank) Pref. score bi (t ) (#rank) Pref. score
1 0.28 (1458) 2.29 -1.16 (1123) 2.60
2 -0.44 (1458) 1.98 1.70 (169) 2.83
3 -0.72 (-) 1.76 -1.13 (-) 1.69
4 -2.13 (1458) 1.47 1.75 (492) 2.17
(a) дu vs. Precision@10 (b) дu vs. Recall@10 5 0.36 (1458) 1.90 0.25 (492) 1.88
Figure 1: Plots of дu values versus Precision@10 and Re-
call@10. Each point is the average of evaluation metrics the examples that our model recommends popular items for the
with regards to the average of дu values of 100 users in de- user who likes popular items and vice versa. bi (t) score represents
scending order of the дu value. item popularity at the period of time t and actual users’ purchase
is shown in bold in Table 2. As can be seen from the user’s pur-
4.4 Experimental Results chase behavior shown in Table 2-(a), the user tends to purchase
Table 1 shows the results of the comparison between our method popular items. Our model learned such purchase behavior from
and conventional approaches. The number of latent factors was the user’s past purchases, and then evaluated the дu value of the
set to 300 and the number of items in a recommendation list to user as 0.69, which means that the user likes popular items. Our
10. In general, time-aware models improve accuracy and reduce model produced popular items for the user, which were ranked
coverage compared with static models. Our model improved both in the top 20. On the other hand, the user in Table 2-(b) selected
accuracy and diversity compared with the baseline in both static items that match the user’s preference without regard to the items’
and time-changing models. Particularly in case of time-varying popularity. Our model captured the tendency from the user’s past
models, our model achieved significant improvement. This indi- purchases and evaluated the user’s дu value as −1.19. Our model
cates that considering temporal item popularity is essential to cap- recommended items that match the user’s preference regardless
ture users’ tendencies. Our model outperformed conventional ap- of their popularity for the user. The preference scores are all high,
proaches in terms of accuracy with the same diversity level. There- while the items’ rankings are various. Therefore, these results indi-
fore, our model effectively captures users’ preference and their ten- cate that our model captured the users’ popularity tendencies and
dencies toward item popularity. recommended personalized items appropriately.
We suppose that the interactions of users with mainstream tastes
are easy to predict. As our model isolates users’ preference from 5 CONCLUSIONS
their tendencies toward item popularity, we can verify the idea In this paper, we proposed a novel approach for counteracting pop-
by analyzing the distribution of accuracy depending on the mag- ularity bias, using MF-based CF incorporating individual users’
nitude of дu values. Figure 1 shows the plots of дu values versus tendencies toward temporal item popularity. Our model isolated
two evaluation metrics: Precision@10 and Recall@10; each point users’ preference from popularity tendency clearly, and considered
is the average of evaluation metrics with regards to the average of temporal item popularity. The experimental results based on a real-
дu values of 100 users in descending order of the дu value. As can world dataset showed the efficacy of our model.
be seen from Fig. 1, both Precision@10 and Recall@10 of the users In future work, we plan to further verify the effectiveness of our
who have large дu values are high. Therefore, this result supports proposed model by using various datasets in different domains or
our assumption. by learning other optimization methods for top-N recommenda-
We also investigated the relation between users’ purchase be- tion. Moreover, as well as item popularity, users’ tendencies toward
havior, which corresponds to their tendencies toward temporal item popularity may change over time. We plan to investigate this
item popularity, and our model’s recommendations. Table 2 shows temporal phenomenon.
Accurate and Diverse Recommendation based on
Users’ Tendencies toward Temporal Item Popularity TempRRS ’17, August 2017, Como, Italy
REFERENCES https://doi.org/10.1145/2507157.2507210
[1] Ruining He, Chen Fang, Zhaowen Wang, and Julian McAuley. 2016. Vista: A [19] Chao-Yuan Wu, Amr Ahmed, Alex Beutel, Alexander J. Smola, and How Jing.
Visually, Socially, and Temporally-aware Model for Artistic Recommendation. 2017. Recurrent Recommender Networks. In Proceedings of the Tenth ACM In-
In Proceedings of the 10th ACM Conference on Recommender Systems (RecSys ’16). ternational Conference on Web Search and Data Mining (WSDM ’17). ACM, New
ACM, New York, NY, USA, 309–316. https://doi.org/10.1145/2959100.2959152 York, NY, USA, 495–503. https://doi.org/10.1145/3018661.3018689
[2] Yifan Hu, Yehuda Koren, and Chris Volinsky. 2008. Collaborative Filtering for [20] Hsiang-Fu Yu, Nikhil Rao, and Inderjit S Dhillon. 2016. Temporal Regular-
Implicit Feedback Datasets. In Proceedings of the 2008 Eighth IEEE International ized Matrix Factorization for High-dimensional Time Series Prediction. In Ad-
Conference on Data Mining (ICDM ’08). IEEE Computer Society, Washington, DC, vances in Neural Information Processing Systems 29, D. D. Lee, M. Sugiyama, U. V.
USA, 263–272. https://doi.org/10.1109/ICDM.2008.22 Luxburg, I. Guyon, and R. Garnett (Eds.). Curran Associates, Inc., 847–855.
[3] Dietmar Jannach, Lukas Lerche, Iman Kamehkhosh, and Michael Jugovac. 2015. [21] Xiangyu Zhao, Zhendong Niu, and Wei Chen. 2013. Opinion-based Collabora-
What Recommenders Recommend: An Analysis of Recommendation Biases and tive Filtering to Solve Popularity Bias in Recommneder Systems. In 24th Interna-
Possible Countermeasures. User Modeling and User-Adapted Interaction 25, 5 tional Conference on Database and Expert Systems Applications (DEXA). 426–433.
(Dec. 2015), 427–491. https://doi.org/10.1007/s11257-015-9165-3
[4] Michael Jugovac, Dietmar Jannach, and Lukas Lerche. 2017. Efficient Opti-
mization of Multiple Recommendation Quality Factors According to Individ-
ual User Tendencies. Expert Syst. Appl. 81, C (Sept. 2017), 321–331. https:
//doi.org/10.1016/j.eswa.2017.03.055
[5] Yehuda Koren. 2008. Factorization Meets the Neighborhood: A Multifaceted Col-
laborative Filtering Model. In Proceedings of the 14th ACM SIGKDD International
Conference on Knowledge Discovery and Data Mining (KDD ’08). ACM, New York,
NY, USA, 426–434. https://doi.org/10.1145/1401890.1401944
[6] Yehuda Koren. 2009. Collaborative Filtering with Temporal Dynamics. In Pro-
ceedings of the 15th ACM SIGKDD International Conference on Knowledge Dis-
covery and Data Mining (KDD ’09). ACM, New York, NY, USA, 447–456. https:
//doi.org/10.1145/1557019.1557072
[7] Qiuxia Lu, Tianqi Chen, Weinan Zhang, Diyi Yang, and Yong Yu. 2012. Serendipi-
tous Personalized Ranking for Top-N Recommendation. In Proceedings of the The
2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intel-
ligent Agent Technology - Volume 01 (WI-IAT ’12). IEEE Computer Society, Wash-
ington, DC, USA, 258–265. http://dl.acm.org/citation.cfm?id=2457524.2457692
[8] Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel.
2015. Image-Based Recommendations on Styles and Substitutes. In Proceedings
of the 38th International ACM SIGIR Conference on Research and Development
in Information Retrieval (SIGIR ’15). ACM, New York, NY, USA, 43–52. https:
//doi.org/10.1145/2766462.2767755
[9] Xia Ning, Christian Desrosiers, and George Karypis. 2015. A Comprehensive
Survey of Neighborhood-Based Recommendation Methods, In Recommender
Systems Handbook. Springer, New York City, New York.
[10] Jinoh Oh, Sun Park, Hwanjo Yu, Min Song, and Seung-Taek Park. 2011. Novel
Recommendation Based on Personal Popularity Tendency. In Proceedings of the
2011 IEEE 11th International Conference on Data Mining (ICDM ’11). IEEE Com-
puter Society, Washington, DC, USA, 507–516. https://doi.org/10.1109/ICDM.
2011.110
[11] Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-
Thieme. 2009. BPR: Bayesian Personalized Ranking from Implicit Feedback.
In Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial In-
telligence (UAI ’09). AUAI Press, Arlington, Virginia, United States, 452–461.
http://dl.acm.org/citation.cfm?id=1795114.1795167
[12] Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2000. Analysis
of Recommendation Algorithms for e-Commerce. In Proceedings of the 2Nd ACM
Conference on Electronic Commerce (EC ’00). ACM, New York, NY, USA, 158–167.
https://doi.org/10.1145/352871.352887
[13] Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-
based Collaborative Filtering Recommendation Algorithms. In Proceedings of the
10th International Conference on World Wide Web (WWW ’01). ACM, New York,
NY, USA, 285–295. https://doi.org/10.1145/371920.372071
[14] Yue Shi, Alexandros Karatzoglou, Linas Baltrunas, Martha Larson, Nuria Oliver,
and Alan Hanjalic. 2012. CLiMF: Learning to Maximize Reciprocal Rank with
Collaborative Less-is-more Filtering. In Proceedings of the Sixth ACM Conference
on Recommender Systems (RecSys ’12). ACM, New York, NY, USA, 139–146. https:
//doi.org/10.1145/2365952.2365981
[15] Harald Steck. 2010. Training and Testing of Recommender Systems on Data
Missing Not at Random. In Proceedings of the 16th ACM SIGKDD International
Conference on Knowledge Discovery and Data Mining (KDD ’10). ACM, New York,
NY, USA, 713–722. https://doi.org/10.1145/1835804.1835895
[16] Yong Kiam Tan, Xinxing Xu, and Yong Liu. 2016. Improved Recurrent Neural
Networks for Session-based Recommendations. In Proceedings of the 1st Work-
shop on Deep Learning for Recommender Systems (DLRS 2016). ACM, New York,
NY, USA, 17–22. https://doi.org/10.1145/2988450.2988452
[17] Yusuke Tanaka, Takeshi Kurashima, Yasuhiro Fujiwara, Tomoharu Iwata, and
Hiroshi Sawada. 2016. Inferring Latent Triggers of Purchases with Considera-
tion of Social Effects and Media Advertisements. In Proceedings of the Ninth ACM
International Conference on Web Search and Data Mining (WSDM ’16). ACM, New
York, NY, USA, 543–552. https://doi.org/10.1145/2835776.2835789
[18] Jason Weston, Hector Yee, and Ron J. Weiss. 2013. Learning to Rank Recommen-
dations with the K-order Statistic Loss. In Proceedings of the 7th ACM Confer-
ence on Recommender Systems (RecSys ’13). ACM, New York, NY, USA, 245–248.