Cross-domain recommender systems aim to generate or enhance personalized recommendations in a target domain by exploiting knowledge (mainly user preferences) from other source domains. Due to the heterogeneity of item characteristics across domains, content-based recommendation methods are difficult to apply, and collaborative filtering has become the most popular approach to cross-domain recommendation. Nonetheless, recent work has shown that the accuracy of cross-domain collaborative filtering based on matrix factorization can be improved by means of content information; in particular, social tags shared between domains. In this paper, we review state of the art approaches in this direction, and present an alternative recommendation model based on a novel extension of the SVD++ algorithm. Our approach introduces a new set of latent variables, and enriches both user and item profiles with independent sets of tag factors, better capturing the effects of tags on ratings. Evaluating the proposed model in the movies and books domains, we show that it can generate more accurate recommendations than existing approaches, even in cold-start situations.
1. INTRODUCTION
Recommender systems [
These benefits rely on the assumption that there are similarities or
relations between user preferences and/or item attributes from
different domains. When such correspondences exist, one way to
exploit them is by aggregating knowledge from the involved
domain data sources, for example by merging user preferences
into a unified model [
In either of the above cases, most of the existing approaches to cross-domain recommendation are based on collaborative filtering, since it merely needs rating data, and does not require information about the users’ and items’ characteristics, which are usually highly heterogeneous among domains.
However, inter-domain links established through content-based
features and relations may have several advantages, such as a
better interpretability of the cross-domain user models and
recommendations, and the establishment of more reliable methods
to support the knowledge transfer between domains. In particular,
social tags assigned to different types of items –such as movies,
music albums, and books–, may act as a common vocabulary
between domains [
In this paper, we review state of the art cross-domain
recommendation approaches that utilize social tags to exploit
knowledge from an auxiliary source domain for enhancing
collaborative filtering rating predictions in a target domain.
4 Amazon e-commerce website, http://www.amazon.com
5 eBay consumer-to-consumer website, http://www.ebay.com
Specifically, we focus on several extensions of the matrix
factorization technique proposed in [
Enrich et al. [
Through a series of experiments in the movies and books domains, we show that the proposed approach outperforms the state of the art methods, and validate the main contribution of this work: A model that separately captures user and item tagging information, and effectively transfers auxiliary knowledge to the target domain in order to provide cross-domain recommendations. The reminder of the paper is structured as follows. In section 2 we review state of the art approaches to the cross-domain recommendation problem, focusing on algorithms based on matrix factorization, and on algorithms that make use of social tags to relate the domains of interest. In section 3 we provide a brief overview of matrix factorization methods for single-domain recommendation, and in section 4 we describe their extensions for the cross-domain recommendation case. In section 5 we present and discuss the conducted experimental work and obtained results. Finally, in section 6 we summarize some conclusions and future research lines.
Cross-domain recommender systems aim to generate or enhance
personalized recommendations in a target domain by exploiting
knowledge (mainly user preferences) from other source domains
[
The knowledge aggregation methods merge user preferences (e.g.
ratings, social tags, and semantic concepts) [
Instead of focusing on sharing user or item latent factors, Enrich et
al. [
The model we propose in this paper follows the same line as
Enrich et al. [
Since the proposed cross-domain recommendation model is built upon a matrix factorization collaborative filtering method, in this section we provide a brief overview of the well-known standard rating matrix factorization technique, and the SVD++ and gSVD++ algorithms, which extend the former by incorporating implicit user feedback and item metadata, respectively.
Matrix factorization (MF) methods [
− − − −
,∈ℛ
+ + + ‖‖ + ‖‖
In this formula, the parameter controls the amount of
regularization to prevent high model variance and overfitting. The
minimization can be performed by using gradient descent over the
set ℛ of observed ratings [
For simplicity purposes, in the following we omit the baseline estimates. They, nonetheless, can be easily considered by adding the term into the rating estimation formulas.
The main motivation behind the SVD++ algorithm, proposed by
Koren [
In this model, user preferences are represented as a combination of explicit and implicit feedback, searching for a better understanding of the user by looking at what items she rated, purchased or watched. For this purpose, additional latent factors are combined with the user’s factors as follows: ̂ = + || ∈ In the previous formula, ∈ ℝ, ∈ ℝ, ∈ ℝ represent user, item, and implicit feedback factors, respectively. is the set of items for which the user provided implicit preference, and is the number of latent features.
Similarly to the SVD algorithm, the parameters of the model can be estimated by minimizing the regularized squared error loss over the observed training data: min ∗,∗,∗
,∈ℛ
− + || + ‖‖ + ‖‖ + ∈ ∈ (4) (5) (1) (2) (3)
Again, the minimization problem can be efficiently solved using stochastic gradient descent.
The gSVD++ algorithm [
The model introduces a new set of latent variables ∈ ℝ for metadata that complement the item factors. This idea combined with the SVD++ algorithm leads to the following formula for computing rating predictions:
̂ = + ||
+ ||
∈
∈
(6)
The set contains the attributes related to item , e.g. comedy
and romance in the case of movie genres. The parameter is set
to 1 when the set ≠ ∅, and 0 otherwise. We note that in the
previous formula, both user and item factors are enriched with
new uncoupled latent variables that separately capture information
about the users and items, leading to a symmetric model with four
types of parameters. Again, parameter learning can be performed
by minimizing the associated squared error function with gradient
descent:
∗,m∗,in∗,∗ ,∈ℛ − + || ∈ + || ∈
(7)
+ ‖‖ + ‖‖ + +
∈ ∈
The use of additional latent factors for item metadata is reported
to improve prediction accuracy over SVD++ in [
In this section, we first describe the tag-based cross-domain
collaborative filtering models presented in [
The main hypothesis behind the models proposed in [
1 || ∈ We note here that if the user has not tagged the item, i.e., = ∅, then the model corresponds to the standard matrix factorization technique. Also, even though the tag factors are only combined with the item factors , the user and item factorization components are not completely uncoupled, since the set still depends on the user .
An improvement over the model was also presented in [
1 ||
∈ Here, the set ⊆ only contains those tags for which the p-value of the abovementioned test is < 0.05. This method was called as UserItemRelTags.
As noted by the authors, the previous methods are useful when the user has tagged but not rated an item. However, these methods do not greatly improve over the standard matrix factorization technique in the cold-start situations where new users or items are considered. Aiming to address this limitation, a last approach was proposed, the ItemRelTags model: ̂ = +
1 ||
∈ (10) Now, the set contains all the relevant tags assigned by the whole community to the item , with possible repetitions. Tags that appear more often contribute with more factors to the (8) (9) prediction, and is the number of times tag was applied to item . In this case, the normalization factor is || = ∑∈ .
We note that the set does not depend on the user, and that the user and item components of the factorization are fully uncoupled. This has the advantage that tag factors can also be exploited in the rating predictions for new users for whom tagging information is not available yet, improving over the standard matrix factorization method. The ItemRelTags model, however, does not take into account the possibility that the user has tagged different items other than the one for which the rating is being estimated. In such cases, it may be beneficial to enrich the user’s profile by considering other tags the user has chosen in the past as evidence of her preferences. In the next subsection, we propose a model that aims to exploit this information to generate more accurate recommendations.
Similarly to the SVD++ algorithm, all of the above models can be trained by minimizing the associated loss function with stochastic gradient descent.
Although the previous recommendation models can successfully transfer tagging information between domains, they suffer from some limitations. The UserItemTags and UserItemRelTags models cannot do better than the standard matrix factorization if the user has not tagged the item for which the rating is being estimated, while the ItemRelTags model does not fully exploits the user’s preferences expressed in the tags assigned to other items. In this paper, we propose to adapt the gSVD++ algorithm by introducing an additional set of latent variables ∈ ℝ that enrich the user’s factors and better capture the effect of her tags in the rating estimation. Specifically, we distinguish between two different sets of tags for users and items, and factorize the rating matrix into fully uncoupled user and item components as follows: ̂ = + 1
1 + || ∈
|| ∈ (11) The set contains all the tags assigned by user to any item. Respectively, is the set of tags assigned by any user to item , and plays the role of item metadata in the gSVD++ algorithm. As in the ItemRelTags model, there may be repeated tags in each of the above tag sets, which we account for by considering the number of times a tag appears in or , respectively. In (11), is the number of items on which the user applied tag , and is the number of users that applied tag to item . As previously, tag factors are normalized by || = ∑∈ and || = ∑∈ , so that factors and do not dominate over the rating factors and for users and items with a large number of tags.
In the proposed model, which we call as TagGSVD++, a user’s profile is enhanced with the tags she used, since we hypothesize that her interests are better captured, and that transferring this information between domains can be beneficial for estimating ratings in the target domain. Likewise, item profiles are extended with the tags that were applied to them, as in the ItemRelTags model.
The parameters of TagGSVD++ can be learned from the observed training data by solving the following unconstrained minimization problem: ∗,m∗,in∗,∗ ,∈ℛ , , ∈, ∈
1 1 1 = ∗,m∗,in∗,∗ ,∈ℛ 2 − + || ∈ + || ∈ (12) + 2 ‖‖ + ‖‖ + ∈‖‖ + ∈‖‖ The factor 1⁄2 simplifies the following derivations with no effect on the solution. As in the previous models, a minimum can be found by stochastic gradient descent. For completeness, in the following we list the update rules of TagGSVD++ taking the derivatives of the error function in (12) with respect to the parameters:
+ ∀ ∈ = − + = − + ∈ where the error term is − ̂. In the training phase, we loop over the observed ratings simultaneously updating the parameters according to the following rules: + ∀ ∈ ← − − + || ∑∈ The learning rate determines to what extent the parameters are updated in each iteration. A small learning rate can make the learning slow, whereas with a large learning rate the algorithm may fail to converge. The choice of both the learning rate and the regularization parameter is discussed later in section 5.3.
We have evaluated the proposed TagGSVD++ algorithm (section 4.2) in a cross-domain collaborative filtering setting, by empirically comparing it with the single-domain matrix factorization methods (section 3) and the state-of-the-art crossdomain recommendation approaches described in section 4.1.
We have attempted to reproduce the cross-domain dataset used in
[
In order to simulate the cross-domain collaborative filtering setting, we have downloaded two publicly available datasets for the movies and books domains. The MovieLens 10M dataset6 (ML) contains over 10 million ratings and 100,000 tag assignments by 71,567 users to 10,681 movies. The LibraryThing dataset7 (LT) contains over 700,000 ratings and 2 million tag 6 MovieLens datasets, http://grouplens.org/datasets/movielens 7 LibraryThing dataset, http://www.macle.nl/tud/LT assignments by 7,279 users on 37,232 books. Ratings in both of the datasets are expressed on a 1-5 scale, with interval steps of 0.5.
Since we were interested in analyzing the effect of tags on rating prediction, we only kept ratings in MovieLens on movies for which at least one tag was applied, leaving a total of 24,564 ratings. Also following the setup done by Enrich et al., we considered the same amount of ratings in LibraryThing, and took the first 24,564 ratings. We note, however, that the original dataset contained duplicate rows and inconsistencies, i.e., some user-item pairs had more than one rating. Hence, we preprocessed the dataset removing such repetitions and keeping only the repeated ratings that appeared first in the dataset’s file. We also converted the tags to lower case in both datasets. Table 1 shows the characteristics of the final datasets.
As mentioned above, we have compared the performance of the proposed model against the single-domain matrix factorization baselines from section 3, and the state-of-the-art tag-based algorithms described in section 4.1. All these methods are summarized next: MF The standard matrix factorization method trained by stochastic gradient descent over the observed ratings of both movies and books domains.
SVD++ An adaptation of MF to take implicit data into account. In our experiments, the set contains all the items rated by user . gSVD++ An extension of SVD++ to include item metadata into the factorization process. In our experiments, we have considered as set of item attributes the tags assigned to item by any user. Note that, as tags are content features for both movies and books, this method is suitable for cross-domain recommendation, since knowledge can be transferred through the metadata (tag) factors. This differs from the proposed TagGSVD++ in that users are modeled as in SVD++ by considering rated items as implicit feedback instead of their tags. Also, normalization of the implicit data factors on the user component involves a square root; see equations (6) and (11).
UserItemTags A method that expands an item ’s profile with latent factors of tags that the target user assigned to . Its parameters are learned by simultaneously factorizing the rating matrices of both source and target domains.
UserItemRelTags A variation of the previous method that only takes relevant tags into account, as determined by a Wilcoxon rank-sum test.
ItemRelTags Instead of tags assigned by the user, this method exploits all relevant tags applied by the whole user community, and is thus able to compute rating predictions even if the user has not tagged the target item.
We evaluated all these recommendation methods in two settings,
using MovieLens as source domain and LibraryThing as target
domain, and vice-versa. In both cases, we evaluated the methods
through 10-fold cross-validation, i.e., we shuffled the target
ratings and split them into 10 non-overlapping folds. In each fold,
we left out one part, 10% of the ratings, as test set to estimate the
performance of the methods. The rest 90% of the ratings were
used as a training set to learn the models, and a validation set to
find the optimal values of the models’ parameters. Specifically,
we randomly chose 80% of these remaining ratings, and combined
them with the source domain ratings to build the models. The final
20% left was used for the validation set to select the best number
of factors , learning rate , and regularization . Figure 1 depicts
the split of the data into training, validation, and test sets.
As in [
Since all the methods are designed for the rating prediction task, we measured their performance as the accuracy of the estimated ratings. Specifically, we computed the Mean Absolute Error (MAE) of each model in the different settings described above: = 1
|ℛ| ,∈ℛ | − ̂| where ℛ contains the ratings in the test set we left out for evaluation.
As previously mentioned, we reserved 20% of the target domain training data in each fold for validating the models and finding the best model parameters, in order not to overestimate the performance of the methods.
For hyperparameter optimization, with each method and sparsity level in the target domain, we performed a grid (stepsize) search on the validation set for the values of the learning rate , the amount of regularization , and the number of latent features . To get an idea of the typical values found for the parameters, Table 2 shows the average best values for each method. From the table, we observe that there is not a large difference in the optimal number of factors and learning rates between configurations. In contrast, we note that the amount of regularization needed for the proposed TagGSVD++ method is relatively large, e.g. compare = 0.036 of TagGSVD++ with = 0.009 of MF. This may be due to the additional set of latent variables for tags that our model uses; more complex models are able to account for greater variance in the data and tend to overfit more easily, thus requiring more regularization. In order to analyze how the available information in the target domain affects the stability of the model, Figure 2 shows the optimal value for the regularization parameter for different sparsity levels. We note that the gSVD++, upon which our model is defined, also introduces additional latent variables and yet requires a lower regularization. We argue that the differences between gSVD++ and TagGSVD++ regularizations are caused by the and sets, see equations (6) with = and (11). In Table 1 we see that, on average, the number of tags applied by a user is much larger than the number of rated items. This results in more variables actually taking part in the rating predictions, and hence in a more complex model that requires more regularization to prevent overfitting.
Once we found the best parameters for each method and sparsity level, we ran the models separately in the test set of each fold. The final performance was estimated as the average MAE over the 10 folds. Figure 3a shows the results obtained using LibraryThing as source domain and MovieLens as target domain. All the differences with respect to the TagGSVD++ algorithm are statistically significant as determined with a Wilcoxon signed rank test at the 95% confidence level. It can be seen that the proposed TagGSVD++ method is able to consistently outperform the rest of the methods for all sparsity levels in the target domain, also in the cold-start setting when only 10%-20% of the ratings are available. We also note that cross-domain methods always achieve better accuracy than single-domain MF, although SVD++ effectively exploits implicit feedback and remains competitive until the 50% sparsity level. Then, as the sparsity decreases, cross-domain models provide greater improvements. This indicates that even if plenty of target domain rating data is available, it is still beneficial to transfer knowledge from the source domain.
The results using MovieLens as source domain and LibraryThing
as target domain are shown in Figure 3b. As before, the difference
in MAE between TagGSVD++ and the rest of the methods is
statistically significant, according to the Wilcoxon signed rank
test with 95% confidence level. Again, TagGSVD++ is the best
performing method for all rating sparsity levels, followed by the
cross-domain methods. We now observe that the values of MAE
are in general larger than in the previous case, which seems to
indicate that the transfer of knowledge is not as effective in this
setting. This observation is in accordance with the results reported
in [
Only 13.81% of the tags in MovieLens are shared in LibraryThing (see Table 1), and thus less latent tag factors learned in the source domain can be used in the target to compute rating predictions. Figure 4 shows the average rating prediction error for users with different amounts of observed ratings and tag assignments, using LibraryThing as source domain and MovieLens as target domain. We see that our model achieves the best improvements in coldstart situations, where few ratings and tag assignments are available in the target domain. We also note that the performance degrades for users with more than 20 ratings (respectively, 100 tag assignments), when enough target domain data is available. Nonetheless, in these cases, TagGSVD++ is still able to exploit the learned tag factors to compute more accurate predictions.
Cross-domain recommendation has potential benefits over traditional recommender systems that focus on single domains, such as alleviating rating sparsity in a target domain by exploiting data from a related source domain, improving the quality of recommendations in cold-start situations by inferring new user preferences from other domains, and by personalizing crossselling strategies to provide customers with suggestions of items of different types.
Despite these advantages, cross-domain recommendation is a
fairly new topic with plenty of research opportunities to explore.
One of the major difficulties that arises is how to link or relate the
different domains to support the transfer of knowledge. Due to the
common heterogeneity of item attributes across domains,
collaborative filtering techniques have become more popular than
content-based methods. However, recent work [
In this paper, we have adapted a novel extension of the wellknown SVD++ algorithm to separately model the effect of user and item tags in the observed ratings. By introducing a new set of latent variables that represent tags in the user profile, our TagGSVD++ method is able to transfer knowledge from a source domain more effectively, providing accurate rating predictions in the target domain, even in cold-start situations. From our experiments in the movies and books domains, we conclude that exploiting additional tag factors, and decoupling user and item components in the factorization process improves the transfer of knowledge and the accuracy of the recommendations. In the future, we plan to further investigate the effect of tags in the quality of recommendations. In particular, we want to study how the recommendation performance depends on the number of shared tags between domains. Increasing the overlap by grouping tags with similar semantics but expressed differently in the domains could favor the transfer of knowledge.
In our experiments we altered the amount of observed rating data
in the target domain, but it would also be interesting to evaluate
the methods varying the number of available ratings in the source
domain. Moreover, we will perform a more exhaustive evaluation
with other datasets including more cross-domain recommendation
methods from the state of the art, such as [
This work was supported by the Spanish Government (TIN201347090-C3-2).