generally referred to as folksonomies, give users the possiWe investigate the problem of item recommendation during bility to annotate items with freely chosen keywords (tags) the first months of the collaborative tagging comCmi unity for easier content retrieval at later points in time [14]. Most teULike. CiteULike is a so-called folksonomy where usoerfs these folksonomy systems recommend suitable tags when have the possibility to organize publications through annao user tags a new item. tations - tags. Making reliable recommendations during the In this paper we consider a problem all new folksono initial phase of a folksonomy is a difficult task, since wienbfosirtes and services offering recommendations encounter mation about user preferences is meager. In order tothiem cold start phase, during which recommendations have prove recommendation results during ctohldis start period, to be made based on very little historical data. Duri we present a probabilistic approach to item recommendthais phase, the similarities between items are hard to calc tion. Our model extends previously proposed models sulcahte as the user-item graph is sparsely, if at all, connect as probabilistic latent semantic analysis (PLSA) by merging However, when turning to tags, we find a higher connecti both user-item as well as item-tag observations into a unified ity. Our objective is to increase the quality of item rec representation. We find that bringing tags into play reducesmendations during this cold start phase by utilizing item the risk of overfitting and increases overall recommendatitoang co-occurrences in conjunction with their user-item coquality. Experiments show that our approach outperformosccurrence counterparts. By doing so, we improve standard other types of recommenders. collaborative filtering models by considering user-given annotations, i.e. tags. Probabilistic latent semantic analysis (PLSA), as introCategories and Subject Descriptors duced by Hofmann in [8], is known for improving reco H.4 [Information Systems Applications]: Information mendation quality in different settings [1]. PLSA assumes a Search and Retrieval lower dimensional latent topic distribution of the observed co-occurrences. These latent distributions group similar items together. We use an extended model of the hybr
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee.
Recommender Systems ’09 New York, New York USA Copyright 2009 ACM X-XXXXX-XX-X/XX/XX ...$10.00.
4http://www.citeulike.org/faq/data.adp 1.1 Related Work responds to the bookmarking of an item by a user and all ob
Recommender systems can be divided into three main cat servations are given by the co-occurrence ImUa.trixUsers egories; collaborative filtering-based, content-based, and so and items are assumed independent given the topic variabl called hybrid systems which combine both. Collaborative fil Z. When applying the aspect model, the probability of a tering approaches base their recommendations solely on citoe m that has been bookmarked by a given user can be com occurrence observations between users and items. Contentp-uted by summing over all latent vaZri:ables based ones, as the name suggests, derive their similarities based on content, i.e. term distributions etc. Hybrid sys P (im|ul) = P (im|zk)P (zk|ul) (1) tems utilize data from both of these models. In folksonomies k tags tend to reflect the content of the tagged item [2], thus For the annotation-based scenario we assume the set of hi even if we do not consider the actual item content dietnselft,opics to be the same as in the item tag co-occurr we group tag-based recommender system together with tohbeservations given bIyT . In compliance with (1), the con content-based ones. ditional probability between tags and items can be writte
The authors of [15] present a hybrid approach to item rec as: ommendation in collaborative tagging communities based on PLSA in which they exploit tags to improve recommenda P (im|tn) = P (im|zk)P (zk|tn). (2) tions on very large datasets. k
Since the introduction of PLSA by Hoffman in [8] it has Following the procedure in [4], we can now merge both mod shown to perform very well in a wide area of topics, elasmboansegd on the common factorP (im|zk) by maximizing the others it continues to outperform multiple other recommelong-likelihood function: dation and decomposition algorithms [1, 16]. A drawback of PLSA is that it does not necessary converge to the global oarpitsiemfuromm t[h5i]s. is Oprneesenwteadyin t[o3, 6o]vwehrceoremtehe anayutheoffrescsthsowthat may L = m α l f (im, ul) log P (im|ul) that multiple training cycles for the same test/train splits proPvaisdte rfeosrearmchore wirtohbiunst threesucltosn.text of recommendation in +(1 − α) n f (im, tn) log P (im|tn) , (3) folksonomies has, until recently, been focused on tag wrehce-re α is a predefined weight for the leverage of each tw ommendation [7, 13]. We apply our extended HyPLSA ap mode model. Using the expectation-maximization (EM) al proach on the task of item recommendation instead. gorithm [4] we subsequently perform maximum likelihood
Another successful approach to recommendation withinparameter estimation for the aspect model. During the ex
folksonomies is the FolkRank algorithm introduced by tpheectation (E) step we begin with calculating the posterior
authors of [
The remainder of this paper is structured as follows. In P (zk|ul, im) = P (im|zk)P (zk|ul) Section 2 we present the algorithms utilized in our exper P (im|ul) iments followed by a description of our tests, dataset and P (im|zk)P (zk|tn) experimental setup in Section 3. We present our results in P (zk|tn, im) = P (im|tn) , Section 4 and draw final conclusions in Section 5.
and then re-estimate parameters in the maximization (M) step according to:
In the following section we describe our HyPLSA approach which is an extended version of the one presented by the authors of [15]. P (zk|tn) ∝ P (zk|ul) ∝ 2.1
Hotho et al. [9] describe a folksonomy as tripartite graph l in which the vertex set is partitioned into three disjoint sets of usersU = {u1, ..., ul}, tagsT = {t1, ...,nt} and itemsI = +(1 − α) n f (tn, im)P (zk|tn, im) (6) {i1, ...,mi }. In [15] Wetzker et al. simplify this model into two bi-partite models; the collaborative filtering ImUodel Based on the iterative computation of the above E and built from the item user co-occurrence coufn(tis, u), and the steps, the EM algorithm monotonically increases the likeli annotation-based modeIlT analogously derived from the co hood of the combined model on the observed data. Usi occurrence total between items and f (tia,gts). In the case the α parameter, our new model can easily be reduced t of social bookmarkingIU becomes a binary matrixf((i, u) ∈ a collaborative filtering, or annotation-based model, simply {0, 1}) since each user can bookmark a given web resbouyrcesettingα to 1.0 or 0.0 respectively. one time only. Given this model, we want to recommeBnedcause of the random initialization of the EM algorith the most interesting new items Ifrtoom userul given his utilized by PLSA, we employ an averaging approach to r or her item history. duce any effects possibly caused by local maximum optimiza
The PLSA aspect model associates the co-occurrence tioofns. Thus, following Equation (1), we repeat Equations (4) observations with a hidden topic varZia=ble{z1, . . . , zk}. to (6)n times for every recommendation and average the In the context of collaborative filtering, an observation pcroorbabilities obtained from Equation (1). Our contribution P (im|zk) ∝ α
f (ul, im)P (zk|ul, im) m m f (ul, im)P (zk|ul, im) f (tn, im)P (zk|tn, im) (4) (5) to the model is presented in Equation (7), where the final, 3.2 Experimental setup averaged, probability is given. To create test and training sets for our algorithms, split each monthly snapshot in two. For all users who
P¯(im|ul) = n Pnn(im|ul) (7) lbeocotekdma8rk0e%d oaft tlheeaisrt i5temitsemass itnhe thteraicnuinrrgentset.snaTphsheotr,emwaein ttrrhheecaieWsodmyepwmreocbeiabnognadhobkteidmlnittoayowriPtkee(m0idrm,e|cublotilyhms)tu.mstfehrnotedmheyuistEeerqamuresaitnioatonptpheuea(nl7d)wte.urdesaieginrFhtiotonergd tihtdbeeamytasendwaitoablethlfreloeumfitsrtesrehettecphwfoeeomredrfrcaomeetraemsnceaadotpnenpcrseraeoqlltauooycwenphnee.tdsltyhEeuovnsuastleeutdstoathteiofosoneprtttrsia.mmtieniesziTatneishgnuegrp.eassrreeatWlsmawteeeitvareeentrldhsyeanvtseehmvrrataoarlguulalegaidnhtseei over all users in 10 independent test runs. 1010 1 2 3 4 5 6 7
months 8 9 10 11 12 // // Figure 3 shows the Prec@10 values for the HyPLS 280315 recommender obtained with an optimal parameter setting 64344 (baalspehda =PL0S.A1, kre=com8m0)e,ndetrhαe =(pur1e,lky =coll1a0b)o,rattihvee pfiultreerliyng 33989 annotation-based PLSA recommender (alpha= 0,k = 80), 4635 the baseline most-popular recommender and the FolkRank 3624 recommender. The figure shows a significant improvement when using the HyPLSA recommender, especially in the early months of the CiteULike. We also see that as dataset grows, and the number of possible items to reco mend increases, the precision values decrease. Nevertheless, the HyPLSA approach delivers significantly better results 24 than the other evaluated approaches.
Figure 4 shows a figure similar to Figure 3, this time number of items, tags, users, Prec@10 values plotted against the number of items in t bookmarks of our data pdeartaset, confirming the observation made earlier.
In Figure 5 we present the relative improvements in pr cision of the HyPLSA approach plotted against the othe ones explored in this paper.
0.07 0.06 0.05 0.02 0.01
The reason for these retsoupltrsovide for better recommendations as the size of dataset
increases. We believe that the reason for the relative i //PLSA3 K=80 α=0.1 provement being higher in the beginning can be traced ba PPLLSSAA33 KK==1800 αα==10..00 to the pattern seen in Figures 1(a) to 1(c) where we initially FMRP see a very much higher density in tag usage compared usage patterns. As the tag usage pattern density become more and more similar to the tag density the recommend tion results of all PLSA (tag, CF and HyPLSA) approaches become similar. 6.