=Paper=
{{Paper
|id=Vol-2431/paper8
|storemode=property
|title=Latent Modeling of Unexpectedness for Recommendations
|pdfUrl=https://ceur-ws.org/Vol-2431/paper8.pdf
|volume=Vol-2431
|authors=Pan Li,Alexander Tuzhilin
|dblpUrl=https://dblp.org/rec/conf/recsys/LiT19
}}
==Latent Modeling of Unexpectedness for Recommendations==
https://ceur-ws.org/Vol-2431/paper8.pdf
Latent Modeling of Unexpectedness
for Recommendations
Pan Li Alexander Tuzhilin
New York University New York University
New York, NY, USA New York, NY, USA
pli2@stern.nyu.edu atuzhili@stern.nyu.edu
Figure 1: Visualization of Latent Convex
Hull. Unexpectedness is defined as the dis- ABSTRACT
tance between new item and latent convex Unexpectedness constitutes an important factor for recommender system to improve user satisfaction
hull generated by all consumed items. and avoid filter bubble issues. Previous methods model unexpectedness in the feature space, making
them difficult to capture the latent, complex and heterogeneous interactions between users and
items. In this paper, we propose to model unexpectedness in the latent space and utilize a latent
convex hull structure to provide unexpected recommendations, as illustrated in Figure 1. Extensive
experiments on two real-world datasets demonstrate effectiveness of latent unexpectedness over
explicit unexpectedness and show that the proposed model significantly outperforms baseline models
in terms of unexpectedness measures while achieving the same level of accuracy.
KEYWORDS
Unexpected Recommendation, Latent Embeddings, Heterogeneous Information Network
INTRODUCTION AND RELATED WORK
Recommender systems have been playing an important role in assisting users in filtering for the best
content and shaping their consumption behaviors. To solve the problem of filter bubbles and boredom
Figure 2: Visualization of Heterogeneous [11, 12], researchers introduce the measure of unexpectedness [10, 14] beyond accuracy, the goal of
Information Network which is to provide novel and not previously seen recommendations. It is positively correlated with
user satisfaction and helps to achieve strong recommendation performance [3–5, 8].
However, one limitation with unexpectedness is that it is defined in the feature space of previously
consumed items, which relies completely on explicit purchase information, and may not work well
ACM RecSys 2019 Late-breaking Results, 16th-20th September 2019, Copenhagen, Denmark
Copyright ©2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International
(CC BY 4.0).
Latent Modeling of Unexpectedness for Recommendations ACM RecSys 2019 Late-breaking Results, 16th-20th September 2019, Copenhagen, Denmark
in the case when purchase actions are sparse or noisy. Besides, it does not include specific user and
item features, neither does it address the latent and complex relationships between users and items.
Furthermore, the distance metric between discrete items is hard to define in the feature space, while
Evaluation Metrics in the latent space we have well-defined euclidean distances. Thus, we cannot model the unexpected
relations efficiently in the feature space.
RMSE & MAE Therefore, though researchers have successfully developed algorithms to improve unexpectedness
Root Mean Square Error and Mean Ab- and other novelty metrics, they always come at a price of losing accuracy performance, as pointed out
solute Error measures the differences in [19]. This severely limits the practical use of the unexpected recommendation, especially in business
between estimated ratings and actual applications where the goal is to increase commercial sales and enhance user satisfaction. Thus, it is
ratings. important to design a novel unexpected recommender to increase novelty of recommendations while
Precision@N & Recall@N keeping the same level of accuracy performance.
Precision is the fraction of the rec- In this paper, we propose to define unexpectedness as the distance metric in the latent space (latent
ommended items that are of high feature and attribute embeddings) rather than feature space (explicit users and items) by utilizing
utility to the user. Recall is the fraction Heterogeneous Information Network Embeddings (HINE) [15, 16] to capture the latent, complex
of the high utility items that are and heterogeneous relations between them. We take the natural closure of convex hull in the latent
recommended to the user. space, and calculate unexpectedness as the euclidean distance from new item embedding to the latent
Unexpectedness convex hull of expected items of the user. The proposed unexpectedness measure is subsequently
Unexpectedness is calculated through combined with estimated ratings for providing recommendations.
equation (3) following our proposed We make the following contributions in this paper. We propose to model unexpectedness in the
definition. latent space by defining the expected set for each user as a latent convex hull generated by item
Serendipity embeddings. We also conduct extensive experiments on two real-world datasets and show that our
Serendipity [8] is computed by model significantly improves novelty measures without losing any accuracy metrics.
RS &P M &U se f ul
Serendipity = RS where
METHOD
RS stands for the recommended items
using the target model, PM stands In prior literature [4], unexpectedness is defined as the distance of recommended item from the
for the recommendation items using closure set of expected items, the latter including items that either previously consumed by the user or
a primitive prediction algorithm and closely related to the consumptions. However, this definition does not include the feature information
USEFUL stands for the items whose for users and items, neither does it consider the latent and heterogeneous interactions between them.
utility is above certain threshold. Thus, it might not serve as a perfect definition, for we cannot manually codify all explicit associations
Diversity and relations between users and items. Also, it is hard to mathematically formalize the expected set
Diversity is computed as the in the feature space, where the distribution of users and items is discrete and unstructured.
averageÍ intra-list distance [17]: On the other hand, heterogeneous information network [16] (HIN) along with latent embedding
D(R) = i ∈R j,i ∈R d(i, j)
Í method [7] provides us with an efficient tool to model users, items and their associated features
Coverage simultaneously by linking the associated features with corresponding entities in the network and
Coverage is computed as the percent- subsequently map them into the latent space. Thus in the paper, we propose to combine those two
age of distinctive recommended items techniques for latent modeling of unexpectedness.
over all distinctive items in the dataset.
Latent Modeling of Unexpectedness for Recommendations ACM RecSys 2019 Late-breaking Results, 16th-20th September 2019, Copenhagen, Denmark
We denote the heterogeneous network as G = (V , E,T ), in which each node v and each link e
Baseline Models
are assigned with specific type Tv and Te . To effectively learn node representations, we enable the
skip-gram mechanism to maximize the probability of each context node c t within the neighbors of v,
SPR [9]
denoted as Nt (v), where we add the subscript t (t ∈ Tv ) to limit the node to a specific type:
Serendipitous Personalized Ranking ex-
tends traditional personalized ranking
Õ Õ Õ
arд max θ loдP(c t |v; θ ) (1)
methods by considering item popular- v ∈V t ∈Tv c t ∈N t (v)
ity in AUC optimization, which makes
To calculate P(c t |v; θ ), we propose to use heterogeneous random walk to generate meta-paths of
the ranking sensitive to the popularity R1 R2
of negative examples. multiple types of nodes in the form of V1 −−→ V2 −−→ V3 · · · Vn wherein R = R 1 ◦ R 2 ◦ · · · Rn defines the
Auralist [18] composite relations between the start and the end of the heterogeneous random walk. The transition
Auralist is a personalized recommenda- probability within each random walk between two nodes is defined as follows:
tion system that balances between the ( C(T ,T )
|N t +1 (Vt )| , (Vt , Vt +1 ) ∈ E
Vt Vt +1
desired goals of accuracy and novelty p(Vt +1 |Vt ) = (2)
simultaneously. 0, (Vt , Vt +1 ) < E
HOM-LIN [4]
where C(TVt ,TVt +1 ) stands for the transition coefficient between the type of node Vt and the type
HOM-LIN is the state-of-the-art unex-
of node Vt +1 . |Nt +1 (Vt )| stands for the number of nodes of type Vt +1 in the neighborhood of Vt . We
pected algorithm that utilizes the com-
apply heterogeneous random walk iteratively to each node and generate the collection of meta-path
bination of estimated ratings and unex-
sequences for the skip-gram mechanism.
pectedness for recommendation.
Note that in previous definition [4], the idea of ‘’closure set” plays an important role. As a natural
DPP [6]
closure in the latent space, latent convex hull of each user is the smallest convex set that contains all
The Determinantal Point Process uti-
the item embeddings that the user has visited. Under this setting, we assume that the expected set
lizes a fast greedy MAP inference ap-
maintains its convexity in the growing process. For example, if a person enjoy rap music and rock &
proach to generate relevant and diverse
roll, she/he shall also have certain expectations on rap-rock music, an innovation combination of those
recommendations.
two genres. Subsequently, we define the unexpectedness as the distance between the embedding
Random
of new item and the latent convex hull generated from the embeddings of all visited items
Random is the baseline model where
of the user:
we randomly recommend items to
users without considering any informa- U nexpu,i = d(i; LC(Ni )) (3)
tion about the ratings, unexpectedness where Ni = (i 1 , i 2 , · · · , i n ) contains the embeddings of items that link to the user in the heterogeneous
and utility. information network. Specifically, the distance function is well defined as the minimal distance from
the given point to all the boundaries of the latent closure. We visualize this definition in Figure 1.
Once we set up the definition of unexpectedness, we perform the unexpected recommendation
using hybrid-utility based collaborative filtering methods that linearly combine estimated ratings
(representing accuracy) with unexpectedness (representing novelty)
U tilityu,i = (1 − α) ∗ EstRatinдu,i + α ∗ U nexpu,i (4)
Latent Modeling of Unexpectedness for Recommendations ACM RecSys 2019 Late-breaking Results, 16th-20th September 2019, Copenhagen, Denmark
The key idea lies in that, instead of recommending the similar items that the users are very familiar with
Dataset Yelp TA as the classical recommenders do, we recommend unexpected and useful items to the users that they
# Review 5,996,996 878,561 might have not thought about, but indeed fit well to their satisfactions. These two adversarial forces of
# Business 188,593 576,689 accuracy and novelty work together to obtain the optimal solution, thus improving recommendation
# User 1,518,169 3,945 performance and user satisfaction. In real practice, the unexpected coefficient α is optimized through
# Sparsity 0.002% 0.039% Bayesian optimization.
Table 1: Descriptive Statistics for the two
Datasets. EXPERIMENTS AND RESULTS
We implement our model on two real-world datasets: the Yelp Challenge Dataset Round 12 [2] and
the TripAdvisor Dataset [1]. The descriptive statistics of these two datasets are listed in Table 1. To
Data Model Unexp Ser Div Cov
FM 0.0326 0.0978 0.0135 0.5369 address the cold-start issue, we filter out users and items that appear less than 5 times in the dataset.
FM+Unexp 0.1122* 0.4793* 0.3798* 0.5904 We measure the recommendation results using accuracy and novelty metrics listed in page 2.
CoCluster 0.0338 0.4595 0.3106 0.5811
CoCluster+Unexp 0.1400* 0.4660* 0.3269* 0.5904 We compare the recommendation performance with and without considering the proposed unex-
SVD 0.0457 0.1352 0.0479 0.5221 pectedness by 5-fold cross-validation experiments using five popular collaborative filtering algorithms
SVD+Unexp 0.0620* 0.1469* 0.0524* 0.5406
NMF 0.0333 0.4954 0.3268 0.5867 including KNN, SVD, CoClustering, NMF and FM [13]. We also report results for baseline unexpected
Yelp NMF+Unexp 0.1390* 0.5469* 0.3430* 0.5904 recommendation models. As shown in Table 2 and 3, when considering unexpectedness in the rec-
KNN 0.0448 0.0977 0.0129 0.5369
KNN+Unexp 0.0610* 0.1165* 0.0259* 0.5406 ommendation process, we obtain significant amount of improvements in terms of Unexpectedness,
SPR 0.0668 0.3720 0.2532 0.5697 Serendipity and Diversity measures, without sacrificing any accuracy performance in RMSE, MAE,
Auralist 0.0663 0.3637 0.2047 0.5457
HOM-LIN 0.0751 0.4329 0.3011 0.5365 Precision and Recall. Therefore, the proposed definition of unexpectedness enables us to provide
DPP 0.0670 0.4488 0.2488 0.5904 unexpected and useful recommendations at the same time. It is crucial for successful deployments of
Random 0.1733 0.4848 0.3763 0.5457
FM 0.0222 0.3979 0.0017 0.1798 unexpected recommendation models in industrial applications. In addition, the proposed approach
FM+Unexp 0.0643* 0.4631* 0.0493* 0.1798 outperforms all previous unexpected recommendation models introduced in page 3, which validates
CoCluster 0.0234 0.3973 0.0015 0.1855
CoCluster+Unexp 0.0652* 0.4619* 0.0471* 0.1807 the superiority of modeling unexpectedness in the latent space over feature space. Finally, we point out
SVD 0.0231 0.3967 0.0006 0.1798 that the proposed model achieves greater improvements on Yelp dataset, which contains richer feature
SVD+Unexp 0.0644* 0.4621* 0.0499* 0.1798
NMF 0.0227 0.3979 0.0010 0.1798 information compared to the TripAdvisor dataset. This observation is in line with our assumption
TA NMF+Unexp 0.0644* 0.4627* 0.0499* 0.1798 that feature information matters in the definition of unexpectedness.
KNN 0.0233 0.3979 0.0019 0.1798
KNN+Unexp 0.0643* 0.4631* 0.0492* 0.1798
SPR 0.0474 0.3593 0.0375 0.1834 CONCLUSION
Auralist 0.0473 0.3462 0.0355 0.1834
HOM-LIN 0.0572 0.3729 0.0411 0.1807 In this paper, we propose to model unexpectedness and user expectations in the latent space, which
DPP 0.0464 0.3245 0.0311 0.1807 makes it possible to capture the latent, complex and heterogeneous relations between users and items,
Random 0.0833 0.4468 0.0650 0.1834
thus significantly improving the performance and practicability of unexpected recommendations.
Table 2: Comparison of recommenda-
We empirically demonstrate that the proposed approach consistently and significantly outperforms
tion performance in novelty measures, ”*”
stands for 95% statistical significance baseline models in terms of unexpected measures without sacrificing the performance of accuracy.
As the future work, we plan to conduct live experiments with real business environment in order to
further evaluate the effectiveness of unexpected recommendations and analyze both qualitative and
quantitative aspects in an online retail setting, especially with the utilization of A/B test.
Latent Modeling of Unexpectedness for Recommendations ACM RecSys 2019 Late-breaking Results, 16th-20th September 2019, Copenhagen, Denmark
REFERENCES
[1] 2019. TripAdvisor Dataset. http://www.cs.cmu.edu/~jiweil/html/hotel-review.html.
[2] 2019. Yelp Challenge Dataset. https://www.yelp.com/dataset/challenge.
[3] Panagiotis Adamopoulos. 2014. On discovering non-obvious recommendations: Using unexpectedness and neighborhood
selection methods in collaborative filtering systems. In Proceedings of the 7th WSDM.
Data Model RMSE MAE Pre@5 Rec@5 [4] Panagiotis Adamopoulos and Alexander Tuzhilin. 2015. On unexpectedness in recommender systems: Or how to better
FM 0.9197 0.6815 0.7699 0.6123
expect the unexpected. ACM Transactions on Intelligent Systems and Technology (TIST) (2015).
FM+Unexp 0.9178 0.6820 0.7700 0.6123
CoCluster 0.9499 0.7140 0.7255 0.5913 [5] Li Chen, Yonghua Yang, Ningxia Wang, Keping Yang, and Quan Yuan. 2019. How Serendipity Improves User Satisfaction
CoCluster+Unexp 0.9504 0.7138 0.7196 0.5864 with Recommendations? A Large-Scale User Evaluation. In The World Wide Web Conference. ACM, 240–250.
SVD 0.9132 0.7071 0.7692 0.5944 [6] Laming Chen, Guoxin Zhang, and Eric Zhou. 2018. Fast greedy MAP inference for Determinantal Point Process to improve
SVD+Unexp 0.9134 0.7076 0.7701 0.5975
recommendation diversity. In Advances in Neural Information Processing Systems. 5622–5633.
NMF 0.9533 0.7181 0.7197 0.5318
Yelp NMF+Unexp 0.9522 0.7154 0.7222 0.5833 [7] Yuxiao Dong, Nitesh V Chawla, and Ananthram Swami. 2017. metapath2vec: Scalable representation learning for
KNN 0.9123 0.7048 0.7687 0.6085 heterogeneous networks. In Proceedings of the 23rd ACM SIGKDD. ACM.
KNN+Unexp 0.9128 0.7051 0.7659 0.6073 [8] Mouzhi Ge, Carla Delgado-Battenfeld, and Dietmar Jannach. 2010. Beyond accuracy: evaluating recommender systems
SPR 1.0351 0.7729 0.7692 0.6188
by coverage and serendipity. In Proceedings of the fourth ACM conference on Recommender systems. ACM, 257–260.
Auralist 1.0377 0.7799 0.7678 0.6000
HOM-LIN 0.9609 0.7447 0.7621 0.6150 [9] Qiuxia Lu, Tianqi Chen, Weinan Zhang, Diyi Yang, and Yong Yu. 2012. Serendipitous personalized ranking for top-n
DPP 1.0288 0.7702 0.7598 0.6012 recommendation. In 2012 WI-IAT. IEEE.
Random 1.4959 1.2456 0.4250 0.3333 [10] Tomoko Murakami, Koichiro Mori, and Ryohei Orihara. 2007. Metrics for evaluating the serendipity of recommendation
FM 1.1105 0.8340 0.6768 0.9590 lists. In Annual conference of the Japanese society for artificial intelligence. Springer, 40–46.
FM+Unexp 1.1275 0.8445 0.7040 0.9656
CoCluster 1.0178 0.7643 0.6845 0.9732 [11] Tien T Nguyen, Pik-Mai Hui, F Maxwell Harper, Loren Terveen, and Joseph A Konstan. 2014. Exploring the filter bubble:
CoCluster+Unexp 1.0285 0.7541 0.6865 0.9703 the effect of using recommender systems on content diversity. In Proceedings of the 23rd WWW. ACM.
SVD 0.9868 0.7533 0.7210 0.9465 [12] Eli Pariser. 2011. The filter bubble: How the new personalized web is changing what we read and how we think. Penguin.
SVD+Unexp 0.9937 0.7517 0.7085 0.9594 [13] Francesco Ricci, Lior Rokach, and Bracha Shapira. 2015. Recommender systems: introduction and challenges. In
NMF 1.0241 0.7709 0.6850 0.9681
TA NMF+Unexp 1.0262 0.7533 0.6881 0.9775 Recommender systems handbook. Springer, 1–34.
KNN 0.9940 0.7531 0.6969 0.9689 [14] Guy Shani and Asela Gunawardana. 2011. Evaluating recommendation systems. In Recommender systems handbook.
KNN+Unexp 1.0001 0.7483 0.6907 0.9763 [15] Chuan Shi, Binbin Hu, Xin Zhao, and Philip Yu. 2018. Heterogeneous Information Network Embedding for Recommenda-
SPR 1.0328 0.8008 0.6395 0.9325 tion. IEEE Transactions on Knowledge and Data Engineering (2018).
Auralist 1.0318 0.7997 0.6460 0.9390
HOM-LIN 1.0298 0.7902 0.6420 0.9418 [16] Yizhou Sun and Jiawei Han. 2013. Mining heterogeneous information networks: a structural analysis approach. Acm
DPP 1.0304 0.8158 0.6264 0.9303 Sigkdd Explorations Newsletter 14, 2 (2013), 20–28.
Random 1.6857 1.3100 0.3238 0.2500 [17] Mi Zhang and Neil Hurley. 2008. Avoiding monotony: improving the diversity of recommendation lists. In Proceedings of
Table 3: Comparison of recommendation the 2008 ACM conference on Recommender systems. ACM, 123–130.
performance in accuracy measures, ”*” [18] Yuan Cao Zhang, Diarmuid Ó Séaghdha, Daniele Quercia, and Tamas Jambor. 2012. Auralist: introducing serendipity into
stands for 95% statistical significance music recommendation. In Proceedings of the fifth ACM WSDM. ACM.
[19] Zainab Zolaktaf, Reza Babanezhad, and Rachel Pottinger. 2018. A Generic Top-N Recommendation Framework For
Trading-off Accuracy, Novelty, and Coverage. In 2018 IEEE 34th ICDE. IEEE.