The new user problem is a recurring problem in memory based collaborative recommender systems (MBCR). It occurs when a new user is added to the system and there are not enough information to make a good selection of the user's neighbours. As a consequence, the recommended items have poor correlation with the user's interests. We addressed the new user problem by observing the user similarity measure (USM). In this paper we present two novelties that address the new user problem : (i) the usage of a personality based USM to alleviate the new user problem and (ii) a method for establishing the boundary of the cold start period. We succesfully used a personality based USM that yielded signi cantly better recommender performance in the period where the new user problem occurs. Furthermore we presented a new methodology for assessing the boundary of the period where the new user problem occurs.
The new user problem is an important issue in memory based collaborative
recommender systems [Adomavicius and Tuzhilin, 2005]. It occurs when a new user
joins the system and there are no (or there are too few) overlapping ratings
to calculate good estimates of user similarities with rating-based user
similarity measures (USM). We will denote this initial period as the cold start period
(CSP). The consequences of being in the CSP are bad rating predictions for
unseen items and thus poor quality of the recommender system. Usually, the
new user problem (NUP) has been addressed by introducing content-based
approaches which resulted in hybrid systems [Adomavicius and Tuzhilin, 2005,
A
We introduced a personality-based USM using the ve factor model (FFM)
in Tkalcic et al. [2009]. The same approach was late
The proposed approach to use a personality-based USM in the NUP allows us to calculate user similarities immediately, without waiting for the user to rate several items. The underlying assumption for choosing personality as the basis for the proposed user similarity measure is that people with similar personalities have similar tastes for products. In psychology, personality is described as a set of factors that account for the majority of between-user variance in emotive, interpersonal, experiential and attitudinal styles [John and Srivastava, 1999].
The second novelty is a statistical method for determining at which point the
new user problem stops occurring. A review of literature showed that authors
either (i) did not set limits for the CSP [Schein et
The new user problem in collaborative ltering recommenders is described as
the period from the moment when a user joins the system to the moment when
there are enough ratings to yield a stable list of neighbours (i.e. users with
similar preferences). We rewrote this description from various sources [Adomavicius
and Tuzhilin, 2005, Schein et
In this section we de ne the boundary of the CSP. Let us have a user u joining the system. The user starts using the system and gives ratings r(u; h) to items h 2 H where H fh1 : : : hJ g, a set of J items. At any given moment the user has given n ratings to n di erent items which yields the set Run = fr1u : : : rnug (1)
The boundary of the new user problem period (the CSP) for the selected user is the number of ratings NuCS after which the system starts to yield stable sets of users. The consequence of a stable set of users is a stable confusion matrix of recommended items. We de ne that the confusion matrix is stable if a sequence of F -measure values, has statistically equivalent means at di erent n.
We choose the F measure as a scalar measure of the confusion matrix. We denote the F measure when n ratings have been used to calculate neighbours as F n. We de ne the CSP boundary as the point N where the means of F values of the sets
RuNJ = fF N : : : F J g
Ru(N 1)J = fF (N 1) : : : F J g (2) are signi cantly di erent.
In Tkalcic et al. [2009] we presented a user similarity measure that takes two vectors bi = (bi1 : : : bi5) and bj = (bj1 : : : bj5) containing the personality values of two users ui and uj and yields the scalar similarity value (3)
We use the proposed user similarity measure to calculate similarities in the CSP. 3
The goal of the experiment was to (i) assess the CSP boundary and to (ii) see whether the personality-based USM performs better in the CSP. We conducted two experiments with the CF recommender system: (i) one with the personality based USM and (ii) one with the rating based USM.
We used the LDOS-PerA -1 [Tkalcic et al., 2010] dataset which contained all data necessary to carry out our experiments. The dataset provided the usage history (i.e. the log of users' interactions) of 52 users consuming 70 content items (a subset of images from the IAPS1 dataset) and giving explicit ratings to each item. The users' task was to assess the images for their computer's wallpaper. The users' personalities vectors b were assessed using the IPIP502 questionnaire. We used the personality based USM as de ned in Eq. 3 to calculate the distances between the users. We calculated the predicted ratings based on the neighbours' ratings using the adjusted Pearson's coe cient as de ned in Kunaver et al. [2007]. We then compared the predicted ratings with the ground truth ratings which yielded the confusion matrix.
We calculated the rating based USM d(ui; uj ) between two arbitrary users ui and uj based on their respective ratings e(u; h) of the overlapping items hm, where m is the index of the items that both the users have rated d(ui; uj ) = s
X(e(ui; hm) m e(uj ; hm))2 (4)
The dataset used in our experiments had a full ratings-items table without missing values with I users and K ratings. To simulate the CSP we determined a usage history path in the form of a random sequence of ratings, for each user separately. We iterated through cold start stages s from one (the user has given only one rating) to K (the user has rated all items) for each user separately. At each stage 1 s K we performed the recommender procedure and calculated the confusion matrix for the observed user u at the observed stage s. We chose the F measure as the performance measure of the recommender system. The 1 http://csea.phhp.ufl.edu/media.html 2 http://ipip.ori.org/New_IPIP-50-item-scale.htm experimental procedure thus yielded a table of F values at di erent stages s 2 f1::Kg and for each user u 2 U . 3.1
Evaluation methodology We compared the performance of the rating based USM and the personality based USM by testing the hypothesis H0 : R = B5 at di erent cold start stages using the t-test. The value R represents the mean F values using the rating based USM and B5 represents the mean F values using the personality based USM.
We determined the position of the CSP boundary by testing the hypothesis H0 : s = s K where s represents the mean F value at stage s and s K represents the mean F value from stages s + 1 to K, where K is the last observed cold start stage. 4
When seeking for the CSP boundary we calculated the p values which are shown in Fig. 1. On the dataset used we observed that p < 0:05 occurs when the cold start stage is s < 6.
We analyzed the CSP by graphing the quality rate of the recommender(F ) versus the number of ratings used. At each cold start stage s the F measures for each user were calculated.
The results of the t test showed that, on the dataset used, the personality based USM yields a signi cantly higher mean of F values than the rating based USM when the number of ratings taken in account for the calculation of the neighbours is lower than 50 (see Fig. 2). When the number of ratings is higher than 50 the means of F values for both similarity measures are not signi cantly di erent at = 0:05. 5
Experimental results showed that the personality based USM performs significantly better than the rating based USM in cold start conditions. A positive outcome is also the fact that the personality USM is statistically equivalent to the rating based USM which makes it a good candidate for a complete replacement of the rating based USM.
However, the results presented here were veri ed only on the speci c dataset and we don't have any ground to conclude that the presented approach is useful also in other domains. We do speculate that hedonistic-content domains would bene t from the presented approach but this should be veri ed as future work.
The main drawback of the personality based USM is the di culty of acquisition of end users' personality parameters. There are two main obstacles in this: (i) it is annoying for the end user to ll in questionnaires and (ii) the acquisition 100 10−1 10−2 100 10−1 10−2 10−3 10−4 10−5 10−6 0 of personality data raises ethical and privacy issues that need to be addressed rst. The progress beyond the state of the art here is the knowledge that personality does account for between-users variance in entertainment applications.
In the lack of existing methodologies for assessing the boundaries of the new user problem we chose a statistical approach. We acknowledge that further investigations should be conducted to determine how to test for the CSP boundary and that these investigations might conclude that a di erent approach is more suitable.
We provided a methodology for the assessment of the new user boundary. The results presented should not be taken for granted and several repetitions of the procedure should be carried out on di erent datasets.
In this paper we have evaluated a personality based USM under cold start conditions. The results showed that the personality based USM performed significantly better than the rating based USM. Furthermore we described a methodology for the assessment of the CSP border. Both novelties are important in the eld of memory based collaborative ltering recommender systems and should be further explored.