Context-aware recommender systems try to adapt to users' preferences across different contexts and have been proven to provide better predictive performance in a number of domains. Emotion is one of the most popular contextual variables, but few researchers have explored how emotions take effect in recommendations especially the usage of the emotional variables other than the effectiveness alone. In this paper, we explore the role of emotions in context-aware recommendation algorithms. More specifically, we evaluate two types of popular context-aware recommendation algorithms - context-aware splitting approaches and differential context modeling. We examine predictive performance, and also explore the usage of emotions to discover how emotional features interact with those context-aware recommendation algorithms in the recommendation process. RecSys'13, October 12-16, 2013, Hong Kong, China. Paper presented at the 2013 Decisions@RecSys workshop in conjunction with the 7th ACM conference on Recommender Systems. Copyright c 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.
Advances in affective computing have enabled recommender
systems to take advantage of emotions and personality, leading to the
development of affective recommender systems (ARS) [
Emotional context was first exploited by Gonzalez et al [
In this paper, we explore the role of emotions by two classes of popular context-aware recommendation algorithms – contextaware splitting and differential context modeling. Since both of these approaches require that the algorithm learn the importance and utility of different contextual features, they help reveal how emotions work in each algorithm and what roles they can play in recommendation.
The purpose of this study is therefore to address the following research questions: 1. Emotional Effect: Are emotions useful contextual variables for context-aware recommendation? 2. Algorithm Comparison: What algorithms are best suited to make use of emotional variables? Do they outperform the baseline algorithms? 3. Usage of Contexts: How do emotional variables compete with other contextual variables, such as location and time? 4. Roles: How can we understand the specific roles emotional variables can play in those context-aware recommendation algorithms? 2.
Gonzalez et al [
As described in [
The remainder of this paper is organized as follows. In Section 3 and Section 4, we formally introduce the two types of recommendation algorithms we are studying, including their capability to capture the role of emotional contexts in recommendation process. Sections 5 and 6 discuss the experimental evaluations and mining the role of emotions through those context-aware recommendation algorithms, followed by the conclusions and future work in Section 7. 3.
Contextual pre-filtering is popular as it is straightforward to
implement, and can be applied with most recommendation techniques.
Item splitting [
To better understand and represent the splitting approaches, consider the following movie recommendation example: In Table 1, there are one user U 1, one item T 1 and two ratings (the first two rows) in the training data and one unknown rating that we are trying to predict (the third row). There are three contextual dimensions – time (weekend or weekday), location (at home or cinema) and companion (friend, girlfriend or family). In the following discussion, we use contextual dimension to denote the contextual variable, e.g. "Location" in this example. The term contextual condition refers to a specific value in a contextual dimension, e.g. "home" and "cinema" are two contextual conditions for "Location". 3.1
Item splitting tries to find a contextual condition on which to split
each item. The split should be performed once the algorithm
identifies a contextual condition in which items are rated significantly
differently. In the movie example above, there are three
contextual conditions in the dimension companion: friend, girlfriend and
family. Correspondingly, there are three possible alternative
conditions: "friend and not friend", "girlfriend and not girlfriend",
"family and not family". Impurity criteria [
Item splitting iterates over all contextual conditions in each context dimension and evaluates the splits based on the impurity criteria. It finds the best split for each item in the rating matrix and then items are split into two new ones, where contexts are eliminated from the original matrix – it transforms the original multidimensional rating matrix to a 2D matrix as a result. Assume that the best contextual condition to split item T1 in Table 1 is "Location = home and not home", T1 can be split into T11 (movie T1 being seen at home) and T12 (movie T1 being seen not at home). Once the best split has been identified, the rating matrix can be transformed as shown by Table 2(a).
This example shows a simple split, in which a single contextual
condition is used to split the item. It is also possible to perform a
complex split using multiple conditions across multiple context
dimensions. However, as discussed in [
UI splitting is a new approach proposed in this paper – it applies item splitting and user splitting together. Assuming that the best split for item and user splitting are the same as described above, the rating matrix based on UI splitting can be shown as Table 2(c). Here we see that both users and items were transformed, creating new users and new items.
Thus, given n items, m users, k contextual dimensions and d
distinct conditions for each dimension, the time complexity for those
three splitting approaches is the same as O(nmkd) [
Context-aware splitting approaches have been demonstrated to improve predictive performance in previous research, but few of those research results report on the details of the splitting process. More specifically, it is useful to know which contextual dimensions and conditions are selected, and the statistics related to those selections. For our purposes, it is possible to explore how emotions interact with the splitting process by the usage of contexts in the splitting process, which may help discover the role of emotions from this perspective. In this paper, we try all three context-aware splitting approaches, empirically compare their predictive performance, and also explore the distributions of the usage of contexts (emotional variables included) for splitting operations.
Differential context modeling (DCM) is a general contextual
recommendation framework proposed in [
We have successfully applied DCM to user-based collaborative
filtering (UBCF), item-based collaborative filtering (IBCF) and
SlopeOne recommendation algorithms in our previous research [
Recall that we separate different functional components from the recommendation algorithm, therefore, how the algorithms relax or weigh contextual variables can tell the role of the contexts in different components. In this paper, we continue to apply DCM to UBCF, where the four components we separate in UBCF can be described as follows: Neighborhood selection UBCF applies the well-known k nearestneighbor (kNN) approach, where we can select the top-k neighbors from users who have rated on the same item i. If contexts are taken into consideration, the neighborhood can be further restricted so that users have also rated the item i in the same contexts. This gives a context-specific recommendation computation. However, the strict application of such a filter greatly increases the sparsity associated with user comparisons. There may only be a small number of cases in which recommendations can be made. DCM offers two potential solutions to this problem. DCR searches for the optimal relaxation of the context, generalizing the set of contextual features and contextual conditions to reduce sparsity. DCW uses weighting to increase the influence of neighbors in similar contexts.
Neighbor contribution The neighbor contribution is the difference between a neighbor’s rating on an item i and his or her average rating over all items. Context relaxation and context weighting can be applied to the computation of ru. This computation is replaced by one in which this average is computed over ratings from a relaxed set of contexts in DCR, or the average rating can be aggregated by a weighted average across similar contexts under DCW.
User baseline The computation of ra is similar to the neighbor’s average rating and can be made context-dependent in the same way.
User similarity The computation of neighbor similarity sim(a; u) involves identifying ratings ru;i and ra;i where the users have rated items in common. For context-aware recommendation, we can additionally add contextual constraints to this part: use ratings given in matching contexts with context relaxation or ratings weighted by contextual similarity using context weighting.
With these considerations in mind, we can derive a new rating
prediction formula by applying DCR or DCW to UBCF. More
details about the prediction equations and technical specifications can
be found in [
One advantage of DCM is that it allows us to explore the role of contexts in each algorithm component. DCM seeks to optimize the contribution of context in each component, and so, the output of the optimization phase, in which contextual dimensions are selected and/or weighted, can reveal the relative importance of different contextual features in different algorithm components. In this paper, we provide comparisons and also explore the role of emotions in these two classes of context-aware recommendation algorithms.
In this section, we introduce the data sets, evaluation protocols and the specific configurations in our experiments. 5.1
We examine context-aware splitting approaches and DCM with the
real-world data set LDOS-CoMoDa, which is a movie dataset
collected from surveys and used by Odic et al [
In our experiments, we use all 12 contextual dimensions, including three emotional contexts: Mood, DominantEmo and EndEmo, and non-emotional contexts, including Location, time, etc. The goal is to compare emotional contexts with others. For example, in DCR, only influential contextual variables for a specific component are selected – whether emotional contexts are selected or not can indicate the significance of their impacts for this component. For a comprehensive comparison, we also performed experiments without emotional contexts to better evaluate the importance of the emotional dimensions.
In this data, there are the three contextual dimensions that contain emotional information. "EndEmo" is the emotional state experienced at the end of the movie. "DominantEmo" is the emotional state experienced the most during watching, i.e. what emotion was induced most times during watching. "Mood" is what mood the user was in during that part of the day when the user watched the movie. Mood has lower maximum frequency than emotions, it changes slowly, so we assumed that it does not change during watching. "EndEmo" and "DominantEmo" contain the same seven conditions: Sad, Happy, Scared, Surprised, Angry, Disgusted, Neutral, where "Mood" only has simple three conditions: Positive, Neutral, Negative. 5.2
To evaluate the performance of context-aware splitting
approaches, we used four splitting criteria described in [
Usually, a threshold for the splitting criteria should be set so that users or items are only be split when the criteria meets the significance requirement. We use an arbitrary value of 0.2 in the tIG case. For tmean, tchi and tprop, we use 0.05 as the p-value threshold. A finer-grained operation is to set another threshold for each impurity value and each data set. We deem it as a significant split once the p-value is no larger than 0.05. We rank all significant splits by the impurity value, and we choose the top first (highest impurity) as the best split. Items or users without qualified splitting criteria are left unchanged.
In DCM, we used the same configuration in our previous work
[
For the evaluation of predictive performance, we choose the root mean square error (RMSE) evaluated using the 5-fold cross validation. The data set is relatively small and some subjects in the survey were required to rate specific movies, thus precision and recall may be not a good metric, but we plan to evaluate them and other metrics in our future work.
We applied DCM only to user-based collaborative filtering. For
splitting approaches, there are more options – we evaluate the
performance of three recommendation algorithms: user-based
collaborative filtering (UBCF), item-based collaborative filtering (IBCF)
and matrix factorization (MF) for each splitting approach.
Koren [
We used the open source recommendation engine MyMediaLite
v3.07 [
For comparison purposes, we choose the well-known
contextaware matrix factorization algorithm (CAMF) [
In this section, the comparisons of predictive performance are introduced first, followed by the discussion of emotional roles discovered in our experiments. 6.1
We use three treatments of the context information – one is the
data with All Contexts where both emotional contexts and
nonemotional variables are included, and the second one is the data
omitting the emotional context dimensions marked by No
Emotions in the table below. The third one is the data with Emotions
Only emitting all non-emotional variables. The overall
experimental results are shown in Table 3, where the numbers in underlined
in italic are the best RMSEs by each approach (i.e. the best one in
2CAMF_CU is a CAMF approach which utilizes the interaction
between contexts and users. [
The comparison of performances of those approaches over the three forms of data can be visualized by Figure 2. We see that including emotions as contexts can improve RMSE compared with the situation we only use non-emotional contexts (i.e. No Emotions). The results differ when we switch our attention to the "Emotions Only" one. In CAMF, it helps achieve the lowest RMSE with Emotions Only data. But, including all of the contextual information yielded the best RMSE for the other approaches: DCM and context-aware splitting algorithms.
1.11 1.08 1.05 1.02 0.99 0.96 0.93 0.9 0.87
CAMF_C CAMF_CI CAMF_CU
DCR
DCW All Contexts
No Emotions
Item
Splitting Emotions Only
User Splitting
This result is not surprising because both DCM and splitting approaches choose among contextual features, deciding which to apply in recommendation. CAMF, on the other hand, uses all available context information in performing its factorization. Without the benefit of feature selection, adding additional features to CAMF may increase noise.
From an overall view, UI splitting has the best RMSE across all data treatments among those context-aware recommendation algorithms. Table 4 shows more details of the predictive performance among the three splitting approaches if we include emotions as the contexts (i.e. using all contexts). The numbers are shown as RMSE values, where the numbers in bold are the best performing RMSE values for each recommendation algorithm across all three splitting approaches. The numbers in underlined in italic are the best RMSE values achieved for the data set using each splitting approach. The numbers in underlined in bold are the global best RMSE for the data set.
These tables show that adding emotions to contexts is able to provide improvement in terms of RMSE, thus answering research question 1 in the affirmative. It also suggests an answer to question 2: that the UI splitting approach outperforms other two splitting approaches if it is configured optimally. In particular, the best RMSE values are achieved by UI splitting using MF as the recommenda
Item Splitting
User Splitting
UI Splitting Generally, MF is the best performing recommendation algorithm. This is not surprising because splitting increases sparsity and MF approaches are designed to handle sparsity data. Because the difference in RMSE is small, we show the boxplot of RMSEs among the best performing item splitting, user splitting and UI splitting approaches (i.e. the configuration as the underlined values in Table 4) in Figure 3. The data in the figure are the 120 RMSE values which comes from the training iterations – different factors (5 N 60, with 5 increment in each step) and training iteration T (10 T 100, with 10 increment in each step). The figure confirms the comparative effectiveness of UI splitting – the box is significantly lower than the other two approaches with the same training iterations. 6.2
As mentioned before, one reason we sought to explore the usage of emotional contexts to discover how emotions interact within context-aware recommendation algorithms. Both splitting and DCM offer insights into how contextual dimensions and features are influential for recommendation. 6.2.1
In splitting approaches, the splitting statistics help discover how contexts were applied in our experiments. In Figure 4 and 5, we show the top selected contextual dimensions for item splitting and user splitting3. The right legend indicates the contextual dimensions, and the y axis denotes the percentage of splits (item splits or user splits) using each dimension. For a clearer representation in the figures, we only show contextual dimensions used on more than 5% (item splitting) or 6% (user splitting) of the recommendations. We do not show results of tIG because this splitting criterion had the worst performance.
In general, the top two dimensions are consistent across those three impurity criteria: EndEmo and Time for item splitting and EndEmo and DominantEmo for user splitting. However, the percentages as y axis in the figures are different, not to mention that the selected condition in each dimension differs too, which results in different performance by using various impurity criteria. In terms of the specific selected emotional conditions, the results are not consistent – the top context dimension for item and user splitting is the same – EndEmo, but the most frequent selected condition in this dimension is "Happy" for item splitting and "Neutral" for user splitting. 3The actual splitting is based on a specific contextual condition in a dimension, but the results of selected contextual conditions are fuzzy, thus the distribution of selected contextual dimensions is explored and reported here.
EndEmo denotes the emotion of the users after seeing the movie,
and this result is consistent with previous work [
In short, the statistics based on splitting approaches reveal the importance of emotions – at least the top first selected contextual dimension is the "EndEmo" for both item splitting and user splitting. 6.2.2
In DCM, the context selection and context weighting can be examined and show, in a detailed way, the contribution of each contextual dimension in the final optimized algorithm. The results are shown in Table 5, where the four components in UBCF were described in the previous section. "N/A" in the table indicates no contextual constrains were placed. For clearer representation, we did not show specific weights of contexts in DCW; instead, we only list variables which were assigned weights above a threshold of 0.7. The weights are normalized to 1, and 0.7 therefore represents a very influential dimension.
We can see in the table that emotional variables are selected in DCR and weighted significantly in DCW and for which components. Emotion is influential for a specific component but may not for other ones. In DCR for example, EndEmo turns out to be influential when measuring user similarities and user baselines, but it is not that significant in computing the neighbor contribution.
It is not surprising that the results from DCW are not fully consistent with ones from DCR. DCW is a finer-grained approach. Emotional variables are assigned to neighbor selection in DCW but not for the same component in DCR, and the specific selected emotions are different too, e.g. it is DominantEmo selected in DCW for user similarity calculation, but it is the EndEmo in DCR. In DCW, the weights for EndEmo and DominantEmo are close to 1 (the weights are all above 0.92) in the component of user baseline, which implies significant emotional influence on this component. Emotional contexts are important in DCM, where it can be further confirmed by Table 3 – the RMSE values are increased if we remove emotions from the contexts.
The result by DCM is useful for further applications, such as affective computing or marketing purposes. Take the results of DCW in Table 5 for example, Mood is influential for selecting neighbors, which implies that if two users rated the same item under the same mood situation, it is highly possible that they are the good neighbor candidates for each other (though neighbor selection also depends on the user similarities). Similarly, DominantEmo is useful to measure user similarities, which infers that users rated items similarly with the same dominant emotions are probably the good neighbors in user-based collaborative filtering.
In conclusion, both context-aware splitting approaches and DCM are able to reveal how emotions interact with algorithms to improve recommendation performance by exploring the usage of contexts in the recommendation or splitting process. More specifically, in context-aware splitting approaches, the percentage of emotional contexts used by item splits or user splits can tell the importance of emotions in distinguishing different user rating behavior. DCM provides a way to see very specifically which emotional contexts are influential for which components in the recommendation process.
Table 6 examines how our research questions are answered for each type of context-aware recommendation. As discussed above, we find that contextual dimensions keyed to emotions are very useful in recommendation (Question 1) and that UI splitting outperforms the DCM and CAMF approaches (Question 2).
Question 3 asks about how emotional dimensions compare with other contextual information. Our results show that emotion-linked context makes an important contribution to context-aware recommendation, although other dimensions are also certainly important. For example, we see that EndEmo is the top selected contextual dimension in item and user splitting, with Time running the second in item splitting, and DominantEmo is the top second selected contextual dimension in user splitting.
Our fourth research question asks about how those two classes of context-aware recommendation algorithms can tell the roles of emotions by usage of contexts. Context-aware approaches are able to infer the roles by the distribution of usage of emotions selected for the splitting process. DCM techniques help answer this question, by pointing out which components make good use of different contextual variables – showing here that DominantEmo is important for calculating the user’s baseline. Note that the neighbor contribution component in DCW is alone in making significant use of non-emotional context dimensions. This is interesting because in this component we are determining which movies to use to compute the neighbor’s baseline for prediction. It appears that the system prefers to use non-emotional considerations in making use of others’ ratings, even though the user’s baseline is computed based on emotional dimensions.
In addition, we can get extra information from our splitting experiments because splitting is performed based on specific contextual conditions. As described above, we found in particular that "Happy" vs not-"Happy" was the important split on the EndEmo dimension for item splitting. Essentially, the algorithm is indicating that there is an important difference between users who feel happy at the conclusion of a given movie and those that do not.
In our future work, we plan to continue our exploration of these algorithms and more data using additional metrics, such as recall and/or normalized discounted cumulative gain. We are also looking at additional data sets to see if similar effects are found with respect to emotions, as well as the empirical comparison among those context-aware recommendation algorithms. We also plan to examine the effects by the correlations between different contextual variables, e.g. how emotional effects change if emotions are significantly dependent with other contexts or features. In addition, it is interesting to explore the association among emotions, user profiles, item features and users’ ratings. For example, user may feel "sad" after seeing a tragedy movie, but the emotion could be "happy" because he or she saw such a good movie even if it is a tragedy. Therefore, which specific emotions will result in a higher rating? the "sad" or the "happy"?