Cross-Domain Recommendation for Large-Scale Data

Abstract

Cross-domain algorithms have been introduced to help improving recommendations and to alleviate cold-start problem, especially in small and sparse datasets. These algorithms work by transferring information from source domain(s) to target domain. In this paper, we study if such algorithms can be helpful for large-scale datasets. We introduce a large-scale cross-domain recommender algorithm derived from canonical correlation analysis and analyze its performance, in comparison with single and cross-domain baseline algorithms. Our experiments in both cold-start and hot-start situations show the e ectiveness of the proposed approach.

INTRODUCTION

Cross-domain recommendation systems are gradually becoming more attractive as a practical approach to improve quality of recommendations. Number of social systems that collect user interaction and preferences in di erent domains is constantly increasing. Accordingly, using information contributed by users in one system to help generate better recommendations in another system in a related domain has become more and more valuable. Especially important in this context is the ability of cross-domain collaborative ltering to soften the cold-start situation by o ering meaningful suggestions at the very start of user interaction with a new domain. Starting with a few proof-of-concept studies [ 1, 2, 6, 10, 19 ], crossdomain recommenders emerged in a sizable stream of research in the recommender systems eld.

Yet, in some sense, the work is still in early stages. While many di erent models have been proposed and explored, the dominating approach to exploring new cross-domain recommendation ideas is to use public datasets that are relatively small in comparison with the full scale of data (items and users) in real-life recommender systems. Full-scale cross-domain datasets are hard to nd, so authors frequently use simulated cross-domain datasets. For example, Iwata and Takeuchi propose a matrix factorization based approach in [ 8 ] where neither users nor items are shared between domains. Although they used a large-scale dataset (using EachMovie, Net ix, and MovieLens), their large-scale dataset is not from a cross-domain system. Rather, this movie rating dataset is divided into random user and item splits. A similar splitting in large-scale movies domain can be seen in [ 15 ]. Moreover, the rare large-scale cross-domain experiment reports in literature focus mostly on content-based cross-domain recommenders [ 4, 13, 18 ]. In [ 12 ], Loni et al. use factorization machines for domains in a large-scale Amazon dataset. In their experiments, better use of within domain information generated better results compared to using cross-domain information. While the current literature show the importance of cross-domain recommender systems, the limitations reviewed above do not allow us to see how cross-domain recommender algorithms scale up.

This paper attempts to ll in the gap of design and evaluation of large-scale cross-domain recommenders by proposing a cross-domain collaborative ltering algorithm and evaluating it using a dataset collected from a multi-domain recommender system, Imhonet. The proposed algorithm, CD-LCCA, is speci cally designed for scalability.

The proposed approach relies on canonical correlation analysis (CCA) [ 7 ] for transferring information from source domain to target domain. CCA has been used in context-aware single-domain recommendation [ 5 ], content-based cross-domain recommendation [ 4 ], and medium-scale cross-domain collaborative ltering [ 17 ]. However, it has not been scaled for large-scale cross-domain collaborative ltering. In this paper, we use a computationally e cient implementation of CCA to model cross-domain recommendations in a large-scale dataset. We present our model in Section 2. We compare the performance of our model with cross-domain and single domain baselines in Section 3, and analyze its cold-start behavior in Section 4. Finally, we present a time performance analysis of the algorithm in Section 5. 2 2.1

LARGE-SCALE CCA-BASED CROSS-DOMAIN ALGORITHM (CD-LCCA) Background

CCA is a multivariate statistical model that studies the interrelationships among sets of multiple dependent variables and multiple independent variables[ 7 ]. Calculating CCA can be very resourceconsuming especially in traditional approaches that should calculate QR-decomposition or singular value decomposition of large data matrices. To avoid this problem, Lu and Foster developed an iterative algorithm that can approximate CCA on very large datasets[ 14 ]. This approach relies on LING, a gradient-based least squares algorithm that can work on large-scale matrices. To compute CCA in L-CCA in [ 14 ], rst a projection of one of the data matrices on a randomly-generated small matrix is produced. Then, a QR-decomposition of this smaller matrix is calculated. After that, CCA is calculated iteratively, by applying LING on the reducedsized QR-decompositions of the original data matrices, in each iteration. Every time after running LING, a QR-decomposition is calculated for numerical stability. Here, we build our large-scale cross-domain recommender algorithm based on L-CCA proposed by Lu and Foster. 2.2

Model

Large scale CCA nds a lower-dimensional representation of each of the input matrices and then calculates the canonical correlation analysis between these two smaller matrices. To base our crossdomain recommender algorithm on LCCA, suppose that we have a n m source domain rating matrix X and a n p target domain rating matrix Y . Here, n represents number of shared users between source and target domains; m shows number of items in source domain; and p shows number of items in target domain. The goal of our model is to estimate user ratings in the target domain (Yi j s), given user ratings in the source domain (Xi j s). We nd the mapping that is between these two domains using LCCA as explained in the following.

Suppose that Xc (n xc ) is a lower dimensional matrix that represents source domain rating matrix X , and Yc (n yc ) is a lower dimensional matrix that represents target rating matrix Y in the LCCA algorithm. Calculating the canonical correlations between Xc and Yc leads us to two canonical variates (XcWxc (n kcca ) and YcWyc (n kcca )) and a diagonal matrix P (kcca kcca ) that shows the canonical correlation between these variates. Using these canonical correlations and variates, we can map Xc to Yc (and vice versa). For example, Yc can be achieved using Equation 1.

Yc = XcWxc PWyTc

Although Equation 1 relates the lower dimensional representations of original source and target domains (Xc and Yc ), we need to map the original source and target matrices (X and Y ) to estimate user ratings in them. To build a relationship between original source and target domain matrices, we rst look at the relationship between each domain matrix and its lower dimensional representation. Without loss of generality, we consider the source domain relationships. Xc is built in the rst step of LCCA by solving an iterative least square problem, having a QR-decomposition in each iteration. Although we loose the mapping information between X and Xc in this iterative process, having both X and nal Xc matrices, we can restore their mapping. We can rewrite X and Xc ’s relationship as in Xc = X M. Here, M is a m cx mapping that projects X into Xc ; and thus:

M = X 1Xc N = Y 1

Yc The same can be applied to nd the mapping of target rating matrices Y and its lower-dimensional representation Yc (Equation 3). (1) (2) (3)

Combining Equations 1, 2, and 3, we can now map between the original source and target rating matrices as presented in Equation 4 and have an estimation of user ratings in the target domain (Yˆ ).

Yˆ = X MWxc PWyc1N 1

When the rating matrix sizes are too large, calculating the multiplications in 4 can be resource-consuming. To resolve this, we take advantage of the fact that »AjB¼ 1C = »A 1C jB 1C¼, and separate the source matrix into multiple smaller matrices, using columnwise partitioning. Then, we perform the multiplication on each of these matrices and eventually join the results together.

Equation 4 gives us the opportunity to relate the source and target domain rating matrices. Based on that, we can estimate the ratings in target domain Y based on ratings in source domain X . In other words, we can estimate user i’s rating on item j from the target domain, given user i’s ratings in the sources domain using Equation 5. Thus, our cross-domain recommender system can suggest the most relevant items to users in the target domain, having user ratings in the source domain. In the following sections, we evaluate our proposed model both in the cold-start and hot-start setting, using a large-scale dataset (4) yˆi; j = Σmq=1Xi;q Σcox=1Mq;o Σkl=cc1aWxco;l Pl;l Σcry=1Wycl;r Nr;1j (5)

Note that the focus of our proposed model is on cross-domain recommenders with shared sets of users across domains. Although some of the research in the area of cross-domain recommender systems is focused on domains with non-overlapping data [ 8, 11, 20, 21 ], the problem of lacking shared users have been a matter of debate [ 3 ]. Some approaches have tried to approach this problem by sharing a subset of users between domains [ 9, 22 ]. We will leave this expansion of the proposed model for future work. 3

DO LARGE-SCALE CROSS-DOMAIN

ALGORITHMS HELP? In our rst set of experiments we study if the proposed crossdomain recommender system is useful in large-scale datasets. In other words, by comparing the cross-domain and single-domain recommendation results, we explore if target domain user data can be enough for achieving good recommendations in large-scale datasets; or if auxiliary information can be helpful. 3.1

Dataset

We use the Imhonet dataset for carrying our experiments in this paper. This is an anonymized dataset obtained from an online Russian recommender service Imhonet.ru. It allows users to rate and review a range of items from various domains, from books and movies to mobile phones and architectural monuments. Imhonet is a true multi-domain system: while it supported di erent domains, each domain was treated almost as an independent sub-site with separate within-domain recommendations. This system also contains many aspects of a social network, including friendship links, blogs and comments. Combination of explicit user feedback (ratings) and diverse domains makes Imhonet very unique and valuable for cross-domain recommendation. We use a dataset that includes Imhonet’s four large domains - books, movies, games, and perfumes. It contains a full set of user ratings (at the time of collection) across four domains.Each rating record in the dataset includes a user ID, an item ID, and a rating value between zero (not rated) and ten. The same user ID indicates the same user across the sets of ratings. Some basic statistics about this dataset are shown in Table 1. To pre-process this dataset we nd shared users across category pairs.

Experiment Setup

To run the experiments, we used a user-strati ed 5-fold crossvalidation setting: 20% of users are selected as test users and the rest of them (80%) are selected as training users. We recommend items to test users given the training data and 20% of their ratings.

Some of the algorithms have parameters that should be selected by cross-validation. To nd the best set of parameters for each algorithm, we select 15% of users as “validation" users and remove 80% of their ratings from the training set. We repeat the experiments 5 times, and report the average performance of algorithms. To measure the performance of algorithms, we use Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE). Although there are other measures, such as rank-based ones, to evaluate recommender systems, we choose these two error measures because of the proposed and baseline algorithm goals: they try to estimate user ratings, instead of optimizing the recommendation rankings. Rank-based measures, such as precision, recall, and nDCG, would not be appropriate for and representative of these recommenders’ performance.

For the single-domain algorithm, we use only the target domain dataset. However, for cross-domain algorithms, we have both source and target datasets. To be able to compare single and crossdomain algorithms, we remove the same set of ratings for all of the algorithms. 0.15 bookNummoevriiecN_guammeberioNco_ukpmNeerufrmuicmereicNb_uopmoekerfNruicumgmeaeNmruiecm_NemurmoicveieriNmc_uopmveieerfNruicmumeNemruicom_veigeraNicmuemgNeaurmimce_eNbrioucpomekerNfruiucmm_emeNroiucvpmieeeNrrfuuicmm_meerNoicuvimeNegruiacmm_egeraNicmupemeNerfuruimcm_eebrNiocoukmNeurimc_ebriocokNumeric

Category Pairs Sorted by CD-CCA RMSE

0.1 bookNummoevriiecN_guammeberioNco_ukpmNeerufrmuicmegreaicNm_uepmNeerufrumicmeeriNcb_uopmoekerNfruicummeemNruiocmv_imeeNroicuvimegNearuimcm_eegNraicumpmeeeNrfruuicmm_emeNroicuvmieeNmruiocm_veigeraNicmpueemrNfeuurmimce_eNbrioucomkeNrguicam_mmeeroNicvuipemeNerufruimcm_eebrNiocoukmNeurimc_ebriocokNumeric

Category Pairs Sorted by CD-CCA MAE see that “books” and “movies” domain pairs have the most number of users and “games” and “perfumes” domains have the least number of common users. The “books” domain has the maximum and “perfumes” domain has the minimum number of items. Also, the “books” domain is among the most sparse domains, while the “perfumes” domain is the least sparse one.

We run the proposed and baseline algorithms on each of these domain pairs. Figures 1 and 2 show RMSE and MAE of algorithms on 12 domain pairs of Imhonet. The reported errorbars represent a 95% con dence interval for errors. As we can see in these gures, the use of cross-domain data with a competitive algorithm originally designed for a single domain doesn’t really help: the single-domain algorithm (SD-SVD) performs better than, or similar to, cross-domain baseline (CD-SVD) in many domains. Only in “movie ! book" and “game ! movie" domain pairs, CD-SVD is signi cantly better than SD-SVD. The domains in these two pairs are semantically closer, compared to other domain pairs. However, CD-LCCA performs signi cantly better than both CD-SVD and SD-SVD in all of the domain pairs. Thus, CD-LCCA is able to see beyond the semantic relationships between domains and capture their latent similarities that may not seem intuitive. Also, we can see that con dence intervals in most of the domain pairs (except for “game ! perfume" and “perfume ! book") are small.

To understand if average error of algorithms are related to each other in di erent domain pairs, we look at RMSE correlations between algorithms that are reported in Table 2. Here, we see that RMSE of CD-SVD and SD-SVD algorithms are highly correlated with each other. However, CD-LCCA’s RMSE does not have any signi cant correlations with the two baseline algorithms’ performance.

Altogether, we conclude that CD-LCCA is helpful in estimating user preferences using auxiliary domain information in large-scale datasets; the baseline cross-domain algorithm that is not designed for this purpose (CD-SVD) may harm the recommendation results rather than helping; error of baseline recommender algorithms are correlated; and CD-LCCA can understand unintuitive, but useful, similarities between domain pairs that are not discovered by CDSVD. 4

DO LARGE-SCALE CROSS-DOMAIN

ALGORITHMS ALLEVIATE COLD-START? One of the major problems in recommender systems literature is the cold-start problem [ 16 ]. Cross-domain recommenders aim to alleviate this problem by transferring target user’s source pro le information for recommendation in target domain. In CD-LCCA, this transfer happens by mapping source and target domains using canonical variates and correlations as in Equation 4. In this section, we investigate the success of such transfer by comparing CD-LCCA, CD-SVD, and SD-SVD’s performance in cold-start setting. To understand how each of these algorithms perform in cold-start setting, we group test users of each dataset based on their target domain source book book book game game game movie movie movie perfume perfume perfume target game movie perfume book movie perfume book game perfume book game movie CD-CCA CD-SVD SD-SVD pro le size. Then, we calculate the error for each group of these users in each of the algorithms. Figure 3 shows number of test users vs. target domain pro le sizes in all of the domain pairs. We can see that most of the test users have a small pro le size (less than 10 items) in the target domain. There are a few users with 100 and more items in their target pro le. To have a better plot, book togame book tomovie 20 40 60 80 100 20 40 60 80 100 book toperfume game tobook we skipped showing these users. Also, there is a concave shape at small (less than 10) target domain pro le sizes. This happens because Imhonet has asked some users to rate at least 20 items, for providing recommendations to them. Since we only use 20% of test user ratings in their target pro les, this increase in the pro le size happens for the pro les that have less than 10 items.

Figures 4 and 5 show the RMSE and MAE of algorithms in the cold-start setting based on target user pro le size, averaged for all of the domain pairs. As we can see in these pictures, in average on all domain-pairs, CD-CCA performs signi cantly better than both of the baselines. Also, the single-domain baseline (SD-SVD) in average performs better than the cross-domain baseline (CD-SVD). In smaller pro le sizes SD-SVD’s error is signi cantly lower than CD-SVD’s error. As the target domain pro le size grows, the errors of two baseline algorithms have no signi cant di erences.

To have a better understanding of cold-start situation in each of the domain pairs, we look at the results of domain-pair combinations separately. Figures 6 and 7 show each algorithm’s cold-start RMSE and MAE in each of the domain pairs. Note that we have plotted the errors for target pro le sizes ranging from one to 100 items. But, in some domain pairs (e.g., “game ! “perfume"), maximum user pro le size is less than 100 and thus the plot is discontinued.

As we can see, for small pro le sizes, in all domain pairs except “game ! perfume" CD-LCCA performs signi cantly better than baseline algorithms. This shows that CD-LCCA can successfully transfer useful information from most source domains to target domain, especially in cold-start situation. For “book” and “movie” 20 40 60 80 100 20 40 60 80 100 game tomovie game toperfume 20 40 60 80 100 20 40 60 80 100 movie tobook movie togame 00 0.6 0.4 0.2 1 0.8 0.6 0.4 0.2 00 0.5 0.45 0.4 0.35 0.3 0.25 0.2 0.150 0.34 0.32 0.3 0.28 0.26 0.24 0.22 0.2 0.18 0.160 CD-CCA CD-SVD SD-SVD target domains the superior performance of CD-LCCA continues in large pro le sizes. But, in “game” and “perfume” target domains performance di erence of algorithms is insigni cant after users have enough items in their target pro le (between 25 and 45 items for di erent domain pairs). There are fewer users with larger pro le sizes in these domains. Thus, we have lower con dence in algorithms’ performance and wider con dence intervals, leading to insigni cant di erences.

Comparing CD-SVD and SD-SVD, we can see that they mostly have similar results. In all experiments with “movie" domain as the source domain, SD-SVD performs signi cantly better than CDSVD from the beginning. But in “game ! movie" and “perfume ! movie", CD-SVD can be signi cantly better than SD-SVD especially in larger pro le sizes. Accordingly, in smaller target pro le sizes not only CD-SVD does not help, but also it can harm recommendation results. This shows that while CD-LCCA can e ciently use the extra source domain information, CD-SVD cannot handle this information e ectively.

Looking at error trends, for some domain pairs (e.g., “movie ! book" and “game ! movie"), we see an initial error increase as the target pro le size grows. Although we expect to see smaller errors, as we have more information from users in target domain, the observed trend is against such expectation. This trend happens in all algorithms including the single-domain baseline (SD-SVD). CD-CCA CD-SVD SD-SVD 50 movie to book

100 50 100 movie to game 50 50 Thus, such behavior cannot be attributed to using extra information

Altogether, we can conclude that not only CD-LCCA can handle extra information from the semantically-related target domain e ciently, but it also can understand the relationship between source and target domains that appear to be unrelated. 5

PERFORMANCE ANALYSIS

In CD-LCCA, calculating large-scale CCA costs O¹N np¹N2 + kpc º + N nk 2º, in which N is number of iterations for least squares; n is number of data points (users); p is the number of items in the target domain; N2 is the number of iterations to compute Yr using gradient descent; kpc is the number of top singular vectors used in LING; and k is the number of components. The multiplications in CD-LCCA depend on the number of nonzero elements in matrices. In the worst case of multiplying dense matrices, the multiplications cost O¹npk + nk 2º. Thus, as a whole, CD-LCCA costs O¹N np¹N2 + kpc º + N nk 2 + npk º. Since kpc 6 k and k 6 p, CD-LCCA costs less than O¹N np¹N2 + k ºº.

In our experiments, we ran all of the algorithms on two similar machines: a MacOS machine with 64GB RAM and two 4-core Intel Xeon, 2:26GHz CPUs and a Linux machine (CentOS) with 64GB RAM and two 4-core Intel Xeon, 2:40GHz CPUs. On average, running CD-LCCA in Matlab on each domain pair took 21210 seconds (close to 6 hours), while running CD-SVD with GraphChi took almost 4 hours. 6

CONCLUSIONS

This work presented a large-scale cross-domain collaborative ltering approach, CD-LCCA. Our experiments on a large-scale useritem rating dataset with 12 domain pairs showed that cross-domain collaborative ltering can be helpful even in large-scale target domains. We saw that CD-LCCA improves recommendation results in both hot and cold-start settings in all domain pairs. But, the baseline cross-domain algorithm helped only in domain pairs with higher semantic similarities. In some cases, adding auxiliary information in the baseline cross-domain algorithm harmed the results. Thus, we concluded that CD-LCCA is able to capture unintuitive relationships between di erent domains, that are not being understood by the baseline algorithms. Our cold-start analysis showed that the proposed model is especially helpful in the cold-start setting. CD-LCCA focuses on domains with shared users. As a follow up to this work, we will expand CD-LCCA to perform cross-domain recommendation in domains with partly-shared, and partly exclusive users.

References

[1] Shlomo Berkovsky , Tsvi Ku ik, and Francesco Ricci . 2007 . Cross-Domain Mediation in Collaborative Filtering . In Proceedings of the 11th international conference on User Modeling (UM '07) . Springer-Verlag, Berlin, Heidelberg.

[2] Shlomo Berkovsky , Tsvi Ku ik, and Francesco Ricci . 2008 . Mediation of user models for enhanced personalization in recommender systems . User Modeling and User-Adapted Interaction 18 , 3 (Aug. 2008 ).

[3] Paolo Cremonesi and Massimo Quadrana . 2014 . Cross-domain Recommendations Without Overlapping Data: Myth or Reality? . In Proceedings of the 8th ACM Conference on Recommender Systems (RecSys '14) . ACM, New York, NY, USA, 297 - 300 .

[4] Ali Elkahky , Yang Song , and Xiaodong He . 2015 . A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems . In International World Wide Web Conference Committee ( IW3C2 ).

[5] Siamak Faridani . 2011 . Using canonical correlation analysis for generalized sentiment analysis, product recommendation and search . In Proceedings of the fth ACM conference on Recommender systems (RecSys '11) . ACM, New York, NY, USA.

[6] Ignacio Fernández-Tobías , Iván Cantador, Marius Kaminskas, and Francesco Ricci . 2012 . Cross-domain recommender systems: A survey of the state of the art . In Spanish Conference on Information Retrieval.

[7] Harold Hotelling . 1936 . Relations Between Two Sets of Variates . Biometrika 28 , 3 /4 ( 1936 ).

[8] Tomoharu Iwata and Koh Takeuchi . 2015 . Cross-domain recommendation without shared users or items by sharing latent vector distributions . In Proceedings of the Eighteenth International Conference on Arti cial Intelligence and Statistics . 379 - 387 .

[9] Arto Klami , Guillaume Bouchard, Abhishek Tripathi, and others. 2014 . Groupsparse Embeddings in Collective Matrix Factorization . In Proceedings of International Conference on Learning Representations (ICLR ) 2014 .

[10] Bin Li . 2011 . Cross-domain collaborative ltering: A brief survey . In Tools with Arti cial Intelligence (ICTAI) , 2011 23rd IEEE International Conference on. IEEE, 1085 - 1086 .

[11] Bin Li , Qiang Yang , and Xiangyang Xue . 2009 . Transfer learning for collaborative ltering via a rating-matrix generative model . In Proceedings of the 26th Annual International Conference on Machine Learning. ACM , 617 - 624 .

[12] Babak Loni , Alan Said, Martha Larson, and Alan Hanjalic . 2014 . ' Free lunch'enhancement for collaborative ltering with factorization machines . In Proceedings of the 8th ACM Conference on Recommender systems. ACM , 281 - 284 .

[13] Yucheng Low , Deepak Agarwal, and Alexander J Smola. 2011 . Multiple domain user personalization . In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM , 123 - 131 .

[14] Yichao Lu and Dean P Foster . 2014 . large scale canonical correlation analysis with iterative least squares . In Advances in Neural Information Processing Systems . 91 - 99 .

[15] Weike Pan , Evan Wei Xiang, and Qiang Yang . 2012 . Transfer Learning in Collaborative Filtering with Uncertain Ratings. . In AAAI.

[16] Denis Parra and Shaghayegh Sahebi . 2013 . Recommender systems: Sources of knowledge and evaluation metrics . In Advanced Techniques in Web Intelligence-2 . Springer, 149 - 175 .

[17] Shaghayegh Sahebi and Peter Brusilovsky . 2015 . It Takes Two to Tango: An Exploration of Domain Pairs for Cross-Domain Collaborative Filtering . In Proceedings of the 9th ACM Conference on Recommender Systems. ACM , 131 - 138 .

[18] Weiqing Wang , Zhenyu Chen, Jia Liu, Qi Qi, and Zhihong Zhao . 2012 . User-based collaborative ltering on cross domain by tag transfer learning . In Proceedings of the 1st International Workshop on Cross Domain Knowledge Discovery in Web and Social Network Mining. ACM , 10 - 17 .

[19] Pinata Winoto and Ti any Tang. 2008 . If You Like the Devil Wears Prada the Book, Will You also Enjoy the Devil Wears Prada the Movie? A Study of CrossDomain Recommendations . New Generation Computing 26 ( 2008 ).

[20] Lei Wu , Wensheng Zhang, and Jue Wang . 2014 . Fusion Hidden Markov Model with Latent Dirichlet Allocation Model in Heterogeneous Domains . In Proceedings of International Conference on Internet Multimedia Computing and Service. ACM , 261 .

[21] Yu Zhang , Bin Cao, and Dit-Yan Yeung . 2010 . Multi-domain collaborative ltering . In Proceedings of the Twenty-Sixth Conference on Uncertainty in Arti cial Intelligence . AUAI Press, 725 - 732 .

[22] Lili Zhao , Sinnojialin Pan , Evanwei Xiang, Erheng Zhong, Zhongqi Lu, and Qiang Yang . 2013 . Active transfer learning for cross-system recommendation .

In Proceedings of the 27th AAAI Conference on Arti cial Intelligence , AAAI 2013 , Bellevue, Washington, USA. 1205 .