Context-aware recommender systems (CARS) go beyond traditional recommender systems, that only consider users' profiles, by adapting their recommendations also to users' contextual situations. Several contextual recommendation algorithms have been developed by incorporating context into recommendation algorithms in different ways. The most effective approaches try to model deviations in ratings among contexts, but ignore the correlations that may exist among these contexts. In this paper, we highlight the importance of contextual correlations and propose a correlationbased context-aware matrix factorization algorithm. Through detailed experimental evaluation we demonstrate that adopting contextual correlations leads to improved performance.
Recommender systems (RS) are an effective ways to alleviate information overload by tailoring recommendations to users’ personal preferences. Context-aware recommender systems (CARS) emerged to go beyond user preferences and also take into account the contextual situation of users in generating recommendations.
The standard formulation of the recommendation problem begins with a two-dimens-ional (2D) matrix of ratings, organized by user and item: Users Items ! Ratings. The key insight of context-aware recommender systems is that users’ preferences for items may be a function of the context in which those items are encountered. Incorporating contexts requires that we estimate user preferences using a multidimensional rating function – R: Users Items Contexts ! Ratings [Adomavicius et al., 2011].
In the past decade, a number of context-aware recommendation algorithms have been developed that attempt to integrate context with recommendation algorithms. Some of the most effective methods, such as context-aware matrix factorization (CAMF) [Baltrunas et al., 2011c] and contextual sparse linear methods (CSLIM) [Zheng et al., 2014b; 2014c], incorporate a contextual rating deviation component into the recommendation algorithms. However, these methods generally ignore contextual correlations that may have bearing on how the rating behavior in different contexts is modeled. In this paper, we highlight the importance of contextual correlation, and propose a correlation-based context-aware matrix factorization algorithm. 2
In context-aware recommender systems, context is usually defined as, “any information that can be used to characterize the situation of an entity [Abowd et al., 1999]”, e.g., time and companion may be two influential contexts in movie domain.
According to the two-part classification of contextual information by Adomavicius, et al. [Adomavicius et al., 2011], we are concerned with static and fully observable contexts, where we already have a set of known contextual variables at hand which remain stable over time, and try to model users’ contextual preferences to provide recommendations.
Several context-aware recommendation algorithms have been developed in the past decade. Typically, context can be taken into account using three basic strategies: pre-filtering, post-filtering and contextual modeling [Adomavicius et al., 2011]. Pre-filtering techniques, such as context-aware splitting approaches [Baltrunas and Ricci, 2014; Zheng et al., 2014a], simply apply contexts as filters beforehand to filter out irrelevant rating profiles. Post-filtering, on the other hand, applies the context as filters to the recommended items after the recommendation process. Or, contexts can be used as filters in the recommendation process, such as differential context modeling [Zheng et al., 2012; 2013]. In contextual modeling, predictive models are learned using the full contextual data, and context information is used in the recommendation process. Most of the recent work, such as CAMF [Baltrunas et al., 2011c], tensor factorization (TF) [Karatzoglou et al., 2010] and CSLIM [Zheng et al., 2014b; 2014c], belong to contextual modeling.
The most effective context-aware recommendation algorithms, such as CAMF and CSLIM, usually incorporate a contextual rating deviation term which is used to estimate users’ rating deviations associated with specific contexts. Alternatively, the contextual correlation could be another way to incorporate contextual information. This idea has been introduced in the context of sparse linear method in our previous work [Zheng et al., 2015], but it has not been explored as part of recommendation models based on matrix factorization. 3
Preliminary: Matrix Factorization and Context-aware Matrix Factorization In this section, we introduce matrix factorization used in recommender systems, as well as the existing research on Context-Aware Matrix Factorization which utilizes the contextual rating deviations. 3.1
Matrix factorization (MF) [Koren et al., 2009] is one of the most effective recommendation algorithm in the traditional recommender systems. Simply, both users and items are represented by vectors. For example, p!u is used to denote a user vector, and !qi as an item vector. The values in those vectors indicate the weights on K (e.g., K = 5) latent factors. As a result, the rating prediction can be described by Equation 1.
r^ui = p!u !qi (1)
More specifically, the weights in p!u can be viewed as how much users like those latent factors, and the weights in !qi represent how this specific item obtains those latent factors. Therefore, the dot product of those two vectors can be used to indicate how much the user likes this item, where users’ preferences on items are captured by the latent factors.
In addition, user and item rating biases can be added to the prediction function, as shown in Equation 2, where denotes the global average rating in the data set, bu and bi represent the user bias and item bias respectively.
r^ui = + bu + bi + p!u !qi (2) 3.2
Consider the movie recommendation example in Table 1. There is one user U 1, one item T 1, and three contextual dimensions – Time (weekend or weekday), Location (at home or cinema) and Companion (alone, girlfriend, family). In the following discussion, we use contextual dimension to denote the contextual variable, e.g. “Location”. The term contextual condition refers to a specific value in a dimension, e.g. “home” and “cinema” are two contextual conditions for “Location”. A context or contextual situation is, therefore, a set of contextual conditions, e.g. fweekend, home, familyg.
More specifically, let ck and cm denote two different contextual situations each of which is composed of a set of contextual conditions. We use ck;l to denote the lth contextual condition in the context ck. For example, assume ck = fweekend, home, aloneg, then ck;2 is the contextual condition “home”. In the following discussion, we continue to use those terms and symbols to describe corresponding contextual dimensions and conditions in the algorithms or equations.
The CAMF algorithm was proposed by Baltrunas et al. [Baltrunas et al., 2011c]. The CAMF rating prediction function is shown in Equation 3.
L + bu + X
j=1 r^uick;1ck;2:::ck;L = Bijck;j + p!u !qi (3)
Assume there are L contextual dimensions in total, then ck = fck;1ck;2:::ck;Lg describes a contextual situation, where ck;j denotes the contextual condition in the jth context dimension. Therefore, Bijck;j indicates the contextual rating deviation associated with item i and the contextual condition in the jth dimension.
A comparison between Equation 2 and Equation 3 reveals that CAMF simply replaces the item bias bi by a contextual
L rating deviation term P Bijck;j , where it assumes that the j=1 contextual rating deviation is dependent on items. Therefore, this approach is named as CAMF CI. Alternatively, this deviation can also be viewed as being dependent on users, which replaces bu by the contextual rating deviation term resulting in the CAMF CU variant. In addition, CAMF C algorithm assumes that the contextual rating deviation is independent of both users and items.
The parameters, such as the user and item vectors, user biases and rating deviations, can be learned by the stochastic gradient descent (SGD) method to minimize the rating prediction errors. In early work [Baltrunas et al., 2011c], CAMF was demonstrated to outperform other contextual recommendation algorithms, such as the tensor factorization [Karatzoglou et al., 2010]. 4
Introducing the contextual rating deviation term is an effective way to build context-aware recommendation algorithms. Our earlier work [Zheng et al., 2014b; 2014c] has successfully incorporated the contextual rating deviations into the sparse linear method (SLIM) to develop contextual SLIM (CSLIM) which was demonstrated to outperform the state-of-the-art contextual recommendation algorithms, including CAMF and tensor factorization.
As mentioned before, contextual correlation is an alternative way to build context-aware recommendation algorithms, other than modeling the contextual deviations. Our recent work [Zheng et al., 2015] has made the first attempt to introduce the contextual correlation (or context similarity) into SLIM. In this paper, we further explore ways to develop correlation-based CAMF.
The underlying assumption behind the notion of “contextual correlation (or context similarity)” is that the more similar or correlated two contexts are, the more similar two recommendation lists for the same user within those two contextual situations should be. When integrated into matrix Time=Weekend Time=Weekday Location=Home Location=Cinema where cE denotes the empty contextual situation – the value in each contextual dimension is empty or “NA”; that is, cE;1 = cE;2 = ... = cE;L = N/A. Therefore, the function Corr(ck; cE ) estimates the correlation between the cE and the contextual situation ck where at least one contextual condition is not empty or “NA”. Note that in Equation 3, the contextual rating deviation can be viewed as the deviation from the empty contextual situation to a non-empty contextual situation.
Accordingly, the user and item vectors, as well as the contextual correlations can be learned based using stochastic gradient decsent by minimizing the rating prediction errors. The loss function is described in Equation 5. Note that this is the general formulation for the loss function, where the term “Corr2” should be specified and adjusted accordingly when the similarity of context is modeled in different ways. We will introduce three context similarity models in the next section.
The remaining challenge is how to represent or model the correlation function in Equation 4. Different representations may directly influence the recommendation performance. In our recent work [Zheng et al., 2015], we considered four strategies: Independent Context Similarity (ICS), Latent Context Similarity (LCS), Multidimensional Context Similarity (MCS) and Weighted Jaccard Context Similarity (WJCS) 1. And those strategies can also be reused for the correlation-based CAMF. The prediction function in Equation 4 and loss function in Equation 5 can be updated accordingly when the correction is represented by different ways.
In this paper, we will not consider WJCS since it only uses the contextual dimensions with the same values, and the correlation in Equation 4 is measured between a context and the empty context. WJCS is therefore not applicable to the correlation-based CAMF, since it only works when the contextual conditions are the same. In the following, we introduce and compare the ICS, LCS and MCS approaches based on CAMF. Note that context correlation or similarity is assumed to be measured between any two contextual situations ck and cm. However, in the correlation-based CAMF, the correlation is actually measured between ck and cE , where cE is the empty context.
1Here, context similarity is identical to context correlation 4.1
An example of a correlation matrix can be seen in Table 2. With Independent Context Similarity, we only measure the context correlation or similarity between two contextual conditions when they lie on the same contextual dimension, e.g., we never measure the correlation between “Time = Weekend” and “Location = Home”, since they are from two different dimensions. Each pair of contextual dimensions are assumed to be independent. In this case, the correlation between two contexts can be represented by the product of the correlations among different dimensions. For example, assume ck is fTime = Weekend, Location = Homeg, and cm is fTime = Weekday, Location = Cinemag, the correlation between ck and cm can be represented by the correlation of <Time = Weekend, Time = Weekday> multiplied by the correlation of <Location = Home, Location = Cinema>, since those two dimensions are assumed as independent.
Assuming there are L contextual dimensions in total, the correlations can be depicted by Equation 6, where ck;l is used to denote the value of contextual condition in the lth dimension in context ck, and the “correlation” function is used to represent the correlation between two contextual conditions, which is also what to be learnt in the optimization. In other words, the correlation between two contexts is represented by the multiplication of the individual correlations between contextual conditions on each dimension.
Corr(ck; cm) =
L Y correlation(ck;l; cm;l) l=1 (6)
These correlation values (i.e., correlation(ck;l; cm;l)) can be learned by the optimization process accordingly. The risk of this representation is that some information may be lost, if correlations are not in fact independent in different dimensions. For example, if users usually go to cinema to see romantic movies with their partners, the “Location” (e.g. at cinema) and “Companion” (e.g. partners) may be significantly correlated as a result. 4.2
As noted earlier, contextual rating data is often sparse, since it is somewhat unusual to have users rate items repeatedly within multiple contextual situations. This poses a difficulty when new contexts are encountered. For example, the correlation between a new pair of contextual conditions <“Time=Weekend”, “Time=Holiday”> may be required in the testing set, but it may not have been learned from the training data due to the sparsity problem. But, the correlation for two existing pairs, <“Time=Weekend”, “Time=Weekday”> and <“Time=Weekday”, “Time=Holiday”>, may have been !"#$%!& !"#$%!& %&'#"($(0,0,4) %&'#"($(0,0,4) /("&0,$(0,6,0) *!(+#&%!& )##*+(,$(2.5,0,0) !!"#$%&(2.5,0,0) '%() )##*#'+$(5,0,0) '%() !!"!#$%(5,0,0) learned. In this case, this representation suffers from the contextual rating sparsity problem. Treating each dimension independently prevents the algorithm from taking advantage of comparisons that might be made across dimensions.
To alleviate this situation, we represent each contextual condition by a vector of weights over a set of latent factors (we use 5 latent factors in our experiments), where the weights are initialized at the beginning and learnt by the optimization process. The dot product between two vectors can be used to denote the correlation between each pair of contextual conditions. As a result, even if the newly observed pair does not exist in the training data, the weights in the vectors representing the two conditions (i.e., “Time=Weekend” and “Time=Holiday”) will be learned and updated by the learning process over existing pairs, and the correlation for the new pair can be easily computed using the dot product. The correlation is given by correlation(ck;l; cm;l) = Vck;l Vcm;l (7) where Vck;l and Vcm;l denote the vector representation for the contextual condition ck;l and cm;l, respectively, over the space of latent factors. We then use the same correlation calculation as shown in Equation 6. We call this approach the Latent Context Similarity (LCS) model. This approach was able to improve the performance of deviation-based CSLIM algorithms too [Zheng, 2015]. In contrast to the independent context similarity approach, LCS provides more flexibility, but it also has the added computational costs associated with learning the latent factors. In LCS, what has to be learnt in the optimization process are the vectors of weights representing each contextual condition. 4.3
Multidimensional Context Similarity (MCS) In the multidimensional context similarity model, we assume that contextual dimensions form a multidimensional coordinate system. An example is depicted in Figure 1.
Let us assume that there are three contextual dimensions: time, location and companion. We assign a real value to each contextual condition in those dimensions, so that each condition can locate a position in the corresponding axis. In this case, a context (as a set of contextual conditions) can be viewed as a point in the multidimensional space. Accordingly, the distance between two such points can be used as the basis for a correlation measure. In this approach, the real values for each contextual condition are the parameters to be learned in the optimization process. For example, the values for “family” and “kids” are updated in the right-hand side of the figure. Thus, the position of the data points associated to those two contextual conditions will be changed, as well as the distance between the corresponding two contexts. Therefore, the correlation can be measured as the inverse of the distance between two data points. In our experiments, we use Euclidean distance to measure the distances, though other distance measures can also be used. The computational cost is directly associated with the number of contextual conditions in the data set, which may make this approach the highest-cost model. Again, the number of contextual conditions can be reduced by context selection. 5
In this section, we present our experimental evaluation and discuss the results. 5.1
We select three context-aware data sets with different numbers of contextual dimensions and conditions. Restaurant data [Ramirez-Garcia and Garca-Valdez, 2014] is comprised of users’ ratings on restaurants in city of Tijuana, Mexico. Music data [Baltrunas et al., 2011a] captures users’ ratings on music tracks in different driving and traffic conditions. The Tourism data [Baltrunas et al., 2011b] collects users’ places of interest (POIs) from mobile applications. The characteristics of these data sets are summarized in Table 3. For more specific information about the contextual dimensions and conditions, please refer to the original papers using those data sets. !"%# !"% We use a five-fold cross validation on our data sets, performing top 10 recommendation task and using precision and mean average precision (MAP) as the evaluation metrics. Precision is defined as the ratio of relevant items selected to number of items recommended in a specific context. MAP is another popular ranking metric which additional takes the ranks of the recommended items into consideration. It is calculated by Equation 8, where M denotes the number of the users, and N is the size of the recommendation list, where P (k) means the precision at cut-off k in the item recommendation list, i.e., the ratio of number of users followed up to the position k over the number k, where m in ap@N denotes the number of relevant items.
N P P (k) k=1 min(m; N ) (8)
We choose the CAMF [Baltrunas et al., 2011c] as baseline in our experiments. We tried all three versions: CAMF CI, CAMF CU and CAMF C, and only present the best performing one which is denoted as CAMF-Dev in the following sections. In addition, we also add tensor factorization (TF) [Karatzoglou et al., 2010] as baseline. 5.3
The results are depicted by Figure 2, where the correlationbased CAMF are represented by three algorithms accordingly: CAMF-ICS, CAMF-LCS and CAMF-MCS.
In terms of the best performing algorithm, CAMF-MCS outperforms the other ones for the restaurant and tourism data sets. It is able to obtain comparable results with CAMFLCS in the music data. Compared with the deviation-based CAMF, the correlation-based CAMF can always outperform CAMF-Dev if the appropriate correlation modeling is applied. For example, in the restaurant data, CAMF-Dev works better than CAMF-ICS in precision, but CAMF-MCS
Restaurant CAMF-Dev CAMF-MCS CSLIM-Dev CSLIM-MCS CAMF-Dev CAMF-MCS 0.1309 0.1586 0.2044 0.2151 0.0210 0.0385 0.1130 0.1276 0.1496 0.1723 0.0226 0.0361 0.2001 0.2765 0.2889 0.2993 0.0474 0.0661 0.2122 0.2840 0.3128 0.3187 0.0542 0.0747 outperforms CAMF-Dev significantly, which states that MCS is the better representation to model contextual correlations than the ICS for this data set.
Tensor Factorization (TF) only works better than some CAMF algorithms in the tourism data set, but CAMF-MCS is still the best one for this data. In TF, contextual dimensions are considered as extra dimensions in addition to the user and item dimensions, and they are assumed as independent with each other, where CAMF algorithms are actually dependent algorithms which further measures either contextual rating deviations or contextual correlations.
In our previous work [Zheng et al., 2014c; 2015], we have incorporated contextual deviations and correlations into SLIM respectively to formulate CSLIM algorithms. Therefore, it is necessary to compare the CAMF and CSLIM, which can be viewed from Table 4, where we only present the best performing deviation and correlation models. From the table, we can see that CSLIM outperforms CAMF significantly when the same strategy (either deviation or correlation modeling) is applied. And CSLIM-MCS works the best in general. It is not surprising, since earlier work [Ning and Karypis, 2011] has demonstrated that SLIM is able to outperform the state-of-the-art traditional recommendation algorithms, including the matrix factorization. Those results based on contextual recommendations further confirms this pattern, since CSLIM outperforms CAMF in view of the Table 4.
In short, those experimental results demonstrate that correlation-based CAMF is able to outperform deviationbased CAMF and the TF algorithm, but the representation for contextual correlation should be carefully selected. Generally, the multidimensional context similarity is the best choice to represent contextual correlations. 6
In this paper, we highlighted the importance of contextual correlation and incorporate this notion into the matrix factorization technique to formulate correlation-based Context-Aware Matrix Factorization (CAMF) algorithm. Our experimental results reveal that correlation-based CAMF is able to outperform the standard deviation-based CAMF and the tensor factorization algorithms. In our future work, we plan to incorporate contextual correlation modeling strategies into more recommendation algorithms, such as the slope one recommender.