Adapt to Emotional Reactions In Context-aware Personalization Yong Zheng Department of Information Technology and Management School of Applied Technology Illinois Institute of Technology Chicago, Illinois, USA yzheng66@iit.edu ABSTRACT attributes as the observed contexts which may change when the user Context-aware recommender systems (CARS) have been devel- performs the same activity repeatedly [38]. For example, the time oped to adapt to users’ preferences in different contextual situa- and location may change every time when a user tries to watch a tions. Users’ emotions have been demonstrated as one of effective movie. The season or trip type may change when a user is going to context information in recommender systems. However, there are reserve a hotel. In addition to these factors, users’ emotional states no work exploring the effect of emotional reactions (or expressions) are one of these dynamic variables. And emotions may change in the recommendation process. In this paper, we assume that users anytime in the process of user interactions with the items or the may give similar ratings even if they present different emotional applications. These emotional information have been demonstrated reactions or expressions on the movies. We further model the traits as effective and influential context in previous research [45, 44]. of emotional reactions and incorporate them into context-aware Emotional reactions or expressions are highly correlated with matrix factorization as regularization terms. Our experimental the traits of user personalities. Personality accounts for the most results based on the LDOS-CoMoDa movie data set validate our important ways in which individuals differ in their enduring emo- assumptions and prove that it is useful to take emotional reactions tional, interpersonal, experimental, attitudinal and motivational into consideration in context-aware recommendations. styles [24]. In the domain of recommender systems, personality can be viewed as a user profile, which may be context-independent and domain-independent. Both emotional information [18, 11, Keywords 34, 45] and user personality [31, 19, 35] have been successfully context, recommendation, emotion, emotional reactions incorporated into recommender systems by existing research. Our previous research [45] has successfully utilized emotional variables as contexts in recommender systems to improve recom- Categories and Subject Descriptors mendation performance. Unfortunately, as far as we know, there H.3.3 [Information Search and Retrieval]: Information filtering, are no research on exploring the effect of emotional reactions Retrieval models; H.1.2 [Models and Principles]: User/Machine or expressions. We believe that users’ emotional reactions or Systems - human information processing expressions are also useful to model users’ preferences or rating behaviors in real practice. For example, two different users may 1. INTRODUCTION AND BACKGROUND give high ratings on a same tragedy drama movie. One of them indicated his or her emotional state as "happy" when finishing the Recommender systems (RS) are an effective way in alleviating movie, because this user thought it was a really good movie. By information overload by tailoring recommendations to users’ per- contrast, another user may express his or her feeling as "sad" since sonal preferences. Context-aware recommender systems (CARS) the user was impressed or moved by the tragedy movie. As a result, take contextual factors (such as time, location, companion, occa- the two users have same rating behaviors on the movie but with sion, etc) into account in modeling user profiles and in generating different emotional reactions or expressions. One of the potential recommendations. For example, users’ choice on movies may be reasons is that different user personalities may result in different very different if the user is going to watch the movie with children ways or habits for users to express their emotions. rather than with his or her partner. Therefore, the users’ rating profiles associated with different (or Context, is usually defined as, "any information that can be even opposed) emotional reactions therefore could be useful to used to characterize the situation of an entity. An entity is assist recommendations. In this paper, we propose to incorporate a person, place, or object that is considered relevant to the emotional reactions (or expressions) as regularization terms in the interaction between a user and an application, including the user context-aware matrix factorization approach, and further explore and applications themselves [12]". In CARS, we view the dynamic its effect on the performance of context-aware recommendations. The following sections are organized as follows: Section 2 introduces related work, including the background of context- aware recommendation, and the role of emotions and personality in recommender systems. Section 3 gives the preliminary description of essential information, such as the LDOS-CoMoDa movie data which contains emotional variables, and the introduction about the CAMF technique. Section 4 discusses our methodology EMPIRE 2016, September 16, 2016, Boston, MA, USA. Copyright held by the author(s). that incorporates the emotional reactions as regularization terms into the CAMF approach. Section 5 describes our experimental to explore the effect of emotional reactions in the context-aware results and discussions, followed by Section 6 which concludes our recommendations. findings and discusses our future work. 3. PRELIMINARY 2. RELATED WORK To further discuss the topics in the context-aware recommendation, One of the goals in the recommender systems (RS) is to assist it is necessary to introduce some terminologies: users’ decision making by providing a list of recommendations. Due to the fact that users’ choice usually varies from time to time Table 1: Sample of a Context-aware Movie Rating Data Set and from context to context, context-aware recommender systems (CARS) [2, 1] are promoted and developed to adapt to users’ User Movie Rating Time Location Companion preferences in different contextual situations. U1 T1 5 Weekday Home Kids In rating-based RS applications, such as movie or book ratings, U1 T1 3 Weekend Cinema Family the standard formulation of the recommendation problem begins U2 T2 3 Weekday Cinema Partner with a two dimensional matrix of ratings, organized by user and U2 T3 4 Weekday Home Kids item: Users × Items → Ratings. The key insight of CARS is that U3 T4 2 Weekend Home Partner users’ preferences on items may be also a function of the context in which those items are encountered. Incorporating contexts requires Table 1 shows an example of context-aware movie data which that we estimate user preferences using a multidimensional rating contains five rating profiles given by three users on four movies function, Users × Items × Contexts → Ratings [1]. in different contextual situations. In our discussions, we will use In the past decade, several context-aware recommendation algo- the term contextual dimension to denote the contextual variable, rithms have been developed. By additionally incorporating context such as "Location", "Time" and "Companion". The term contextual information, these algorithms have been demonstrated to be useful condition refers to a specific value in a contextual dimension, to improve recommendation performance in numerous domains, e.g. "Home" and "Cinema" are two contextual conditions for the such as e-commerce [28, 15], movies [33, 26, 10], music [3, 16], dimension "Location". Context or contextual situation therefore restaurants [30, 27], travels [39, 8], educational learning [37], refers to a combination of contextual conditions, e.g., {Weekday, mobile applications [4, 6], and so forth. The context variables Home, Kids}. adopted in those applications are domain-specific ones. And the Next, we introduce the LDOS-CoMoDa movie data 1 which is most widely used context information are the time of the day, the a data set with multiple contextual dimensions including several day of the week, and location information which can be easily emotional variables. We also introduce context-aware matrix captured from ubiquitous environment, such as Web logs, mobile factorization which is a popular algorithm in CARS and we use devices, sensors. it as a base algorithm in this paper. It is well known that human decision making is subject to both rational and emotional influences [14]. The field of affective 3.1 LDOS-CoMoDa Data Set computing takes this fact as basic to the design of computing In the domain of context-aware recommendation, there are very systems [29]. The role of emotions in recommender systems limited number of data sets available for public research, not to was recognized by the research community as early as 2005 [23], mention the data that contains emotional variables. The LDOS- giving rise to research in emotion-based movie recommender CoMoDa data set [21] introduced below is one of the data sets that systems [18] and the impact of emotions in group recommender was collected from user surveys, and can be used for this type of systems [23, 11]. This results in the highlight of research on research in this paper. The data has 2291 ratings (rating scale is affective recommender systems [34] which have been proved to be 1 to 5) given by 121 users on 1232 items within 12 contextual useful on recommendation performance in several domains, such dimensions. The description of the contextual dimensions and as music [22, 32, 9] and movies [7, 25, 18]. conditions can be described by Table 2. Emotional states, accordingly, are also viewed and used as contexts in recommender systems. Shi et al. [33] mined the mood Table 2: List of Context Information in the LDOS-CoMoDa Data similarity to assist context-aware movie recommendation. Odic, et al. [26] identified the significant contributions by emotional Dimension Contextual Conditions variables compared with other contextual factors in the LDOS- Time Morning, Afternoon, Evening, Night CoMoDa movie rating data. Mood information can also be used Daytype Working day, Weekend, Holiday for television and video content recommendation [36]. Baltrunas, Season Spring, Summer, Autumn, Winter Location Home, Public place, Friend’s house et al. [3] adopted mood as context to assist context-aware music Weather Sunny / clear, Rainy, Stormy, Snowy, Cloudy recommendation. The role of emotions in context-aware recom- Companion Alone, Partner, Friends, Colleagues, Parents, Public, Family mendation is summarized in [45, 44] which helps additionally endEmo Sad, Happy, Scared, Surprised, Angry, Disgusted, Neutral discover insights about why and where emotional states play an domEmo Sad, Happy, Scared, Surprised, Angry, Disgusted, Neutral important role in the recommendation process. Mood Positive, Neutral, Negative Emotional states are usually dynamic and may change from time Physical Healthy, Ill Decision Movie choices by themselves or users were given a movie to time. Based on the introduction about the affective recommender Interaction First interaction with a movie, Nth interaction with a movie systems [34], the emotional information in three stages may be useful: entry stage (i.e., before the activity), consumption stage Among these 12 contextual dimensions, there are three ones (i.e., during the activity) and exit stage (i.e. after the activity). that can be considered emotional dimensions: endEmo, domEmo In this case, emotional reactions can be captured across these and mood. "endEmo" is the emotional state experienced at the three stages. As introduced previously, users may present different end of the movie (i.e., emotion in the exit stage). "domEmo" is emotional reactions, but actually they leave the same or similar 1 ratings on the items. In this paper, we make the first attempt LDOS-CoMoDa data set, http://www.ldos.si/comoda.html the emotional state experienced the most during watching (i.e., emotion in the consumption stage). "mood" is the emotion of   N the user during that part of the day when the user watched the X 1 2 λ X 2 2 2 2 min  err + ( B + bi + ||pu || + ||qi || ) movie (i.e., emotion in the entry stage). "EndEmo" and "domEmo" B∗ ,b∗ ,p∗ ,q∗ r∈R 2 2 j=1 u,cj contain the same seven conditions: Sad, Happy, Scared, Surprised, (3) Angry, Disgusted, Neutral, while "mood" only has three simple Afterwards, the algorithm is able to learn the corresponding conditions: Positive, Neutral, Negative. parameters by minimizing the squared errors in prediction. The Context selection is usually performed before we apply any loss function as shown in Equation 3 is a composition of squared context-aware recommendation algorithms. We’d like to retain error and regularization terms which are used to alleviate the the most influential context dimensions, since irrelevant ones may overfitting problems, where ruic1 c2 ...cN is the real and known introduce noises in the data and further hamper the recommenda- rating given by user u on item i in context c1 c2 ..cN , and λ is the tion accuracy. Based on the statistical selection method introduced regularization rate used in the optimization process. By stochastic in [26], we only use 7 out of the 12 contextual dimensions in our gradient descent, we are able to learn the parameters iteratively and experiments: time, daytype, location, companion and the three finally achieve the best performing CAMF_CU model. emotional variables. CAMF is an effective algorithm and it is able to alleviate the data The three emotional variables (i.e., mood, domEmo and en- sparsity to some extent. We choose CAMF_CU because we are dEmo) describe users’ affective states during the user interactions going to explore the correlation between users and their emotional with the movies in terms of three stages respectively: entry stage, reactions, which requires a user-specific context-dependent model. consumption stage and exit stage as introduced in [34]. In other The same thing can also happen to other algorithms which explore words, mood can be viewed as current context before the user intersections or the dependency between users and contexts, such starts watching the movie. By contrast, domEmo and endEmo can as the CSLIM_CU approach [40]. indicate future emotional states during the user’s interactions with In the next section, we will introduce how to incorporate the the activity of movie watching. These future status can also be emotional reactions as regularization terms to CAMF_CU. viewed as contexts too if we interpret them as user intents. For example, a user is feeling sad now, and he or she wants to select a movie to watch in order to be happy. In this example, "sad" is 4. METHODOLOGY the current user mood, and "happy" can be viewed as user’s future In this section, we introduce our methodology of how to incorpo- emotional state, such as in the domEmo or endEmo. rate emotional reactions into context-aware recommender systems. 4.1 Problem Statement 3.2 Context-aware Matrix Factorization Recall that we assume that the different emotional reactions or One of the most popular context-aware recommendation algo- expressions can be used to model users’ rating behaviors. For 0123ÿ0123ÿ0125 rithms is the one built upon matrix factorization, namely, the example, assume two users gave a high rating on a same tragedy context-aware matrix factorization (CAMF) approach [5]. There drama movie. One of them indicated his or her emotional state are different variants of CAMF, here we introduce the CAMF_CU as "happy" when finishing the movie, because this user thought approach which incorporate a user-personalized contextual rating it was a really good movie. But another user may express his bias into matrix factorization. More specifically, the rating predic- or her feeling as "sad" since it is a tragedy movie. The same 6789 ÿ9 tion function by CAMF_CU can be described by Equation 1. thing may also happen to the domEmo in addition to the endEmo. The emotional reactions or expressions in this paper, refer to the ÿÿ N X different values in the dimension domEmo and/or endEmo in the r̂uic1 c2 ...cN = µ + Bu,cj + bi + pTu qi (1) LDOS-CoMoDa data. ÿÿ! j=1 Figure 1 presents the distribution of rating counts in each Assume there are totally N contextual dimensions. c1 c2 ..cN is emotional state. Note that "Unknown" indicates the missing value 6"#9# ÿ9 used to denote a contextual situation, where c1 indicates the value in the LDOS-CoMoDa. We can observe that Neutral and Happy of contextual condition in the 1st context dimension. r̂uic1 c2 ...cN are the most two common emotional expressions in both domEmo $%ÿ therefore represents the predicted rating for user u on item i in and endEmo. the situation c1 c2 ..cN . The prediction function is composed of 1000 &''2ÿ 800 four components: the global mean rating µ, item rating bias bi , N 700 ($')ÿ P the aggregated contextual rating bias Bu,cj , and user-item 600 9:;<ÿ#=ÿ789 > j=1 interaction represented by the dot product of a user vector and item 500 ()ÿ 400 vector, pTu qi . pu is the user vector represented by a set of latent factors, and qi is the item vector represented by the same set of 00 *$)ÿ factors. pu can tell how much the user u likes those latent factors, 300 while qi indicates how the item i obtains these factors. Therefore, 200 (+)ÿ the dot product function is used to estimate how much the user will 100 like this item. 0 $% &''2 ($') () *$) (+) ,2 -./ ,2ÿ The term Bu,cj is the estimated contextual rating bias for user u in context condition cj . It is used to denote how user u’s rating is )?@? )@? -./ÿ deviated in each contextual condition. Figure 1: Distribution of Rating Counts in Each Emotional State err = ruic1 c2 ...cN − r̂uic1 c2 ...cN (2) 0123ÿ0123ÿ0125 67898ÿ9899 ÿ!!ÿ"#$%&' 0ÿ!!ÿ"#$%&'  (  )  9ÿ89 89ÿ89 *++8,ÿÿ-./. *++8,ÿÿ-/. 2ÿ34'$#$56ÿ"#$%&' 8ÿ96&#$56ÿ"#$%&' )) 7 1 7) 89ÿÿ-./. 89ÿÿ-/.  9ÿÿ-./.  9ÿÿ-/. Figure 2: Distribution of Unusual Emotional Reactions in the LDOS-CoMoDa Data Furthermore, we’d like to learn the unusual case to see whether problem can be summarized as how to incorporate these emotional users present different emotional reactions in this data. An unusual reactions into existing recommendation algorithms. More specif- case could be two situations: 1). a user leaves a negative rating, but ically, we want to explore the approach to incorporate them into expresses positive emotional states in either domEmo or endEmo; the CAMF approach. There are three questions we are particularly 2). a user gives a positive rating while he finally indicates a negative interested in: emotion in either domEmo or endEmo. To explore these unusual cases, we need to define which ratings • How to fuse this emotional reactions into CAMF? and which emotions are positive or negative. In our experiments, we simply view a rating as a positive one if the rating is no less • Does it work by providing improvements? than 4; otherwise, the rating is negative. In terms of the emotional • Which emotional reaction is more effective? The reactions states, we only consider "Happy" and "Surprised" as positive ones, based on domEmo or endEmo? while other emotional states are negative. A simple statistics about unusual cases in this data can be depicted by Figure 2. First of all, 66.5% of the ratings are positive 4.2 Regularization by Emotional Reactions ones as shown in subfigure a). Based on the subfigure b), we First of all, how the user reacts on the movies in terms of emotional can observe that 36.9% of all the rating records are unusual cases status is dependent with what type of movies the user is watching. (i.e., the two situations mentioned above) based on the domEmo In this case, we additionally use movie genre information in the variable, while it is 33.4% in the endEmo variable. This may tell LDOS-CoMoDa data and aggregated users’ ratings for each movie that domEmo could be more effective and useful than endEmo in type. A sample of the aggregated data can be shown in Table 3. modeling users’ emotional reactions. The subfigure c) and d) further describe the two unusual situa- Table 3: An Example of Aggregated Rating Matrix tions among the positive and negative ratings respectively. In the piece of profiles with positive ratings, 43.4% of them are associated User Genre Rating Time domEmo endEmo with negative emotions in domEmo – many more than the cases in U1 Action 5 Weekday Sad Happy endEmo. It is not surprising, since the theme or the genre of the U1 Drama 3 Weekend Sad Sad movie will affect user’s dominating emotions during the process of U2 Cartoon 3 Weekday Happy Angry movie watching. For example, users may feel horrible or scared U2 Drama 3 Weekday Angry Happy when watching a horrible movie, but finally leave a positive rating U3 Action 2 Weekend Sad Sad since it is a good movie. On the other hand, in terms of the records with negative ratings, there are no significant differences for the In Table 3, we replace the column of item by movie genre unusual cases between domEmo (24%) and endEmo (25%) based to construct a new rating matrix. We will use the same 7 on the observations subfigure d). Recall that, there are many more contextual dimensions introduced previously. Note that in the positive ratings than the negative ones in this data. Therefore, it LDOS-CoMoDa data we do not know what the movie genre is, seems that users may express more unusual emotional reactions in since the genre was encoded as numbers in this data. domEmo rather than in endEmo. We suspect that the emotional Afterwards, we can fuse an emotional dimension (either endEmo reactions in domEmo may leave more influential impact on our or domEmo) into the user dimension to create a two-dimensional proposed recommendation models. rating matrix. Let’s take the domEmo for example, the converted The underlying assumption in our proposed approach is that rating matrix can be described by Table 4. user’s emotional reactions or expressions on the future emotional Specifically, we fuse the values in domEmo into the user column states (e.g., domEmo and/or endEmo) can be used to improve rec- to create new users. The new user is represented by a combination ommendations, since they may indicate similar user tastes even if of original user ID and value in the domEmo, and we name those the emotional reactions are different or even opposed. The research new users as emotional users. Meanwhile, we eliminate the other Table 4: Converted Two-Dimensional Rating Matrix the emotional state in domEmo, since it is not necessary to be the same value as cm . But they should be the contextual condition in User, domEmo Genre Rating the same dimension (i.e., the mth dimension). U1, Sad Action 5 As mentioned previously, more similar two emotion users are, U1, Sad Drama 3 their ratings on the items (with same genre) should be similar. U2, Happy Cartoon 3 In our CAMF_CU model, it can be derived that user’s contextual U2, Angry Drama 3 rating deviations in this emotional variable (i.e., the mth contextual U3, Sad Action 2 variable) should be similar. Namely, Bu,cm and Bv,cm+ should be very close. We add the squared difference of these two deviations (e.g., Equation 5) as the regularization term in Equation 4. contextual dimensions from the rating matrix. In this case, we can Additionally, how close the two contextual rating deviations are build a matrix factorization model based on this converted two- should be dependent with the similarity of two emotional users. dimensional rating matrix. And then we are able to calculate the In this case, the regularization term is weighted by the similarity similarity between emotional users based on the cosine similarity between two emotional users. We name this term as "emotional of each two vectors which represent emotional users. For example, regularization term" in this paper. we can measure how similar the "U1, Sad" to "U2, Angry" based Recall that our assumption is that the emotional users should on their co-ratings on the movies with the same genre information. be similar because two difference users have similar ratings even Theoretically, we can use the item information (e.g., item if their emotional reactions are different. It can also tell that the ID) instead of the movie genre in the rating matrix, but it will two users actually share something in common, so we assume increase data sparsity. We use movie genre information only for there should also be a similarity between two users to some two reasons: On one hand, using movie genre is based on our extent. Therefore, we are able to additionally incorporate a assumptions that users’ different emotional reactions depend on "user regularization term" to build a finer-grained recommendation the movie genre and user’s emotional reactions, for example, user model, where the loss function can be shown in Equation 6. Again, may express as happy or sad on a tragedy movie. On the other the user regularization is also weighted by the similarity between hand, it is able to alleviate the rating sparsity in the converted two- two emotional users. dimensional rating matrix so that we can obtain more reliable user similarities. We have tried to use item ID, but emotional users  N   1 err 2 + λ 2 + b2 2 2 P have very few co-ratings on the items, which results in worse X  2 2 ( Bu,c j i + ||pu || + ||qi || )  min  j=1   β recommendation performance compared with that when we use B∗ ,b∗ ,p∗ ,q∗  r∈R  + 2 P Sim((v, cm+ ), (u, cm )) × (reg_user + reg_emo)   v,cm+ ∈K genre information only. Note that we use domEmo as an example (6) in Table 4, while we can also have the same process based on the variable endEmo. reg_user = ||pu − pv || 2 (7) In short, the emotional users should be similar if they have Based on those two different loss functions, we are able to build similar ratings on the movies with same genre information, even two new CAMF approaches by incorporating the emotional reac- if the original users have different emotional reactions on domEmo tions as the regularization terms. We can learn the corresponding or endEmo. For example, the ratings given by "U1, Sad" and "U2, parameters based on the gradient decent accordingly. Note that the Angry" are all 3-star on the drama movies shown in the Table 4. performance of the models may also depend on the number of K- Therefore, U1 with dominating emotion as "Sad" may share similar nearest neighbors used in the algorithm. In our experiments, we set user tastes with U2 with dominating emotion as "Angry" to some different values to explore the best options in these parameters. extent. Accordingly, we are able to create a regularization term based on the similarity of contextual users. The new loss function can be 5. EXPERIMENTS shown as Equation 4, where β is the regularization rate for the new In this section, we introduce our evaluation settings and experimen- regularization terms. tal results, as well as our findings.  N  5.1 Evaluation Protocols  1 err 2 + λ 2 + b2 2 2 P X  2 2 ( Bu,c j i + ||pu || + ||qi || )  We employ a 5-folds cross-validation on the LDOS-CoMoDa data min  j=1   (4) β B∗ ,b∗ ,p∗ ,q∗  r∈R  + 2 P Sim((v, cm+ ), (u, cm )) × reg_emo   set. Namely, we split the rating profiles into 5 folds and perform v,cm+ ∈K 5 rounds evaluations. For each round, one of the fold will be used 2 as testing set, and the other 4 folds of data will be used as training reg_emo = (Bu,cm − Bv,c ) (5) m+ data. We build our recommendation models based on the training We will use the same function shown in Equation 1. In addition, set and evaluate the results according to the ground truth inferred we incorporate a new regularization term in Equation 4 compared from the testing set. with the loss function described by Equation 3. We use CAMF_CU approach as baseline, and compete its More specifically, we use m to denote the index of an emotional recommendation performance with the CAMF_CU models with variable (i.e., either domEmo or endEmo). Take domEmo for different regularization terms. We use the CAMF_CU approach example, m indicates the position of domEmo in the list of implemented in the open-source toolkit, CARSKit [41], to perform contextual dimensions, thus cm is used to express user’s emotional the evaluations. state in domEmo. According, "u, cm " is the emotional user More specifically, we evaluate the recommendation performance (introduced as Table 4), and we use K to denote the top-K nearest based on the rating prediction and top-10 recommendation tasks. neighbor of emotional user "u, cm " based on the user similarity In the rating prediction task, we use mean absolute error (MAE) calculated based on the matrix factorization model built upon the as evaluation metric. We also further examine the statistical converted two-dimensional rating matrix. Namely, "v, cm+ " is one difference of MAE among different algorithms based on paired t- of the identified top-K nearest neighbors. We use cm+ to denote test at a 95% confidence level. In the top-10 recommendation, We 0.73 0.72 Precision NDCG 0.012 0.012 0.71 0.7 0.01 0.01 MAE 0.69 0.008 0.008 Precision NDCG 0.68 0.006 0.006 0.67 0.004 0.004 0.66 0.002 0.002 0.65 0 0 CAMF_CU domEmo_B domEmo_B,u endEmo_B endEmo_B,u CAMF_CU domEmo_B domEmo_B,u endEmo_B endEmo_B,u (a) Results on MAE (b) Results on Top-10 Recommendation Figure 3: Experimental Results on the Rating Prediction and Top-10 Recommendation Tasks adopt precision as the relevance metric and Normalized Discounted endEmo_B, u outperforms endEmo_B (30.1% improvement on Cumulative Gain (NDCG) [20] as the ranking metric. More precision, and 18.5% on NDCG). specifically, precision is calculated as the ratio of relevant items As mentioned before, the number of selected neighbors in selected to the number of items recommended (i.e., 10 in our our models may impact the recommendation performance. We experiment). NDCG is a measure from information retrieval, where present the impact by the number of neighbors in the finer-grained positions are discounted logarithmically. CAMF_CU approaches with two regularization terms, as shown by the Figure 4. Simply, we vary the number of neighbors from 10 to 80 with an increment of 10 on each step. The best number of 5.2 Experimental Results neighbors should be around 40 to 50 in this data set. It is essential First of all, we present our results based on the rating prediction to examine different number of neighbors to find out the optimal task in Figure 3(a). We use CAMF_CU to denote the original selection for each recommendation model. approach without emotional or user regularization terms. Our Finally, the experimental results help us identify that the domEmo approaches introduced in this paper are built upon CAMF_CU is more useful and effective to be adopted than using endEmo. This approach and they can be generated based on either domEmo or finding is consistent with our previous analysis on the unusual cases endEmo. We evaluate the performances by them individually. shown in Figure 2. It makes sense since the emotional status during We use "domEmo_B" to represent the model using domEmo for the process of movie watching may be very different than their emotional regularization, i.e., cm denotes the emotional state in emotions at the end. For example, a user may feel horrible if he domEmo in Equations 4. By contrast, "domEmo_B, u" is used to or she is watching an adventure movie, but finally he or she might denote the finer-grained model described in Equation 6 which con- feel happy since it is a good movie. tains both emotional and user regularization terms. Accordingly, "endEmo_B" and "endEmo_B, u" are the two recommendation 5.3 Discussions models by using endEmo to generate the regularization terms. Why emotional reactions or expressions can be reused to improve Based on the results shown in Figure 3(a), our proposed ap- the recommendation performance? As we mentioned before, one proaches only using the emotional regularization term can help of the potential reasons is that the different emotional reactions obtain lower MAE. All of these improvements are statistically are caused by the traits in different user personalities – users may significant based on the paired t-test. When we try to use both express their emotional states or reactions in different ways. It emotional and user regularization terms, it is able to further has been well studied that the emotional expression has strong improve prediction accuracies. However, the improvement by correlations with user personality, especially in the areas of psy- endEmo_B,u fails the paired t-test compared with the endEmo_B chology and social science. For example, the correlation between approach. The best performing model in the rating prediction task emotional expression and personality can be used to assist health is domEmo_B,u, where we apply emotional and user regularization care [13]. Harker, et al. [17] found that individual differences in terms at the same time, and these regularization terms are generated positive emotional express were linked to personality stability and based on the emotional reactions by domEmo. development across adulthood. However, there are no applications We show the top-10 recommendation results based on precision of using personality inferred from emotional reactions or expres- and NDCG in Figure 3(b). The bars present results based on sions to further serve real-world applications, such as recommender precision at top-10 recommendation, the curve tells the results systems. In this paper, we make our attempts to explore the impacts in NDCG. We can observe similar patterns shown in the rating of emotional reactions or expressions in the recommender systems, prediction task: first, we see that the CAMF_CU models with our especially in the context-aware personalization. regularization terms are able to outperform the original CAMF_CU approach in both precision and NDCG. This finding confirms that incorporating emotional regularization terms inferred from users’ 6. CONCLUSIONS AND FUTURE WORK emotional reactions is helpful to improve performance of context- In this paper, we believe that users may place similar ratings even aware recommendation. if they may have different emotional reactions or expressions. We Furthermore, we can observe the finer-grained model with addi- propose to incorporate the corresponding regularization terms in tional user regularization term contributes to obtain more improve- the CAMF_CU approach to assist context-aware recommendation. ments. For example, domEmo_B,u works better than domEmo_B Our findings based on the experimental results over the LDOS- (19.6% improvement on precision, and 18.2% on NDCG), and CoMoDa movie data demonstrate that modeling user’s emotional Results by domEmo_B Results by domEmo_B,u Precision NDCG Precision NDCG 0.0025 0.01 0.0025 0.012 0.002 0.008 0.002 0.01 0.008 Precision Precision 0.0015 0.006 0.0015 NDCG NDCG 0.006 0.001 0.004 0.001 0.004 0.0005 0.002 0.0005 0.002 0 0 0 0 10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80 the number of neighbors the number of neighbors Results by endEmo_B Results by endEmo_B,u Precision NDCG Precision NDCG 0.0015 0.008 0.002 0.01 0.006 0.0015 0.008 0.001 Precision Precision 0.006 NDCG NDCG 0.004 0.001 0.004 0.0005 0.002 0.0005 0.002 0 0 0 0 10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80 the number of neighbors the number of neighbors Figure 4: Impact by the Number of Neighbors reactions is helpful to improve recommendation performance. The [6] M. J. Barranco, J. M. Noguera, J. Castro, and L. Martínez. A results also reveal that domEmo is better than endEmo to generate context-aware mobile recommender system based on the regularization terms in this data set. And the finer-grained location and trajectory. In Management intelligent systems, model by additionally incorporating user regularization is able to pages 153–162. Springer, 2012. offer further improvements. [7] S. Benini, L. Canini, and R. Leonardi. A connotative space One of our future work is to incorporate these regularization for supporting movie affective recommendation. Multimedia, terms based on different emotional reactions to more context-aware IEEE Transactions on, 13(6):1356–1370, 2011. recommendation models. It is interesting to examine the similar [8] M. Braunhofer, M. Elahi, and F. Ricci. Sts: A context-aware approach in the similarity-based context-aware recommendation mobile recommender system for places of interest. In UMAP algorithms [43, 42] so that we can learn the similarities of not only Workshops. Citeseer, 2014. the emotional users but also the emotion themsevles. We will also [9] R. Cai, C. Zhang, C. Wang, L. Zhang, and W.-Y. Ma. try to explore the effect of emotional reactions in other applications Musicsense: contextual music recommendation using (such as music) rather than the movie domain. emotional allocation modeling. In Proceedings of the 15th international conference on Multimedia, pages 553–556. 7. REFERENCES ACM, 2007. [1] G. Adomavicius, B. Mobasher, F. Ricci, and A. Tuzhilin. [10] P. G. Campos, I. Fernández-Tobías, I. Cantador, and F. Díez. Context-aware recommender systems. AI Magazine, Context-aware movie recommendations: an empirical 32(3):67–80, 2011. comparison of pre-filtering, post-filtering and contextual [2] G. Adomavicius, R. Sankaranarayanan, S. Sen, and modeling approaches. In E-Commerce and web technologies, A. Tuzhilin. Incorporating contextual information in pages 137–149. Springer, 2013. recommender systems using a multidimensional approach. [11] Y. Chen and P. Pu. Cofeel: Using emotions to enhance social ACM Transactions on Information Systems (TOIS), interaction in group recommender systems. In Alpine 23(1):103–145, 2005. Rendez-Vous (ARV) 2013 Workshop on Tools and Technology [3] L. Baltrunas, M. Kaminskas, B. Ludwig, O. Moling, F. Ricci, for Emotion-Awareness in Computer Mediated A. Aydin, K.-H. Lüke, and R. Schwaiger. Incarmusic: Collaboration and Learning, 2013. Context-aware music recommendations in a car. In EC-Web, [12] A. K. Dey. Understanding and using context. Personal and volume 11, pages 89–100. Springer, 2011. ubiquitous computing, 5(1):4–7, 2001. [4] L. Baltrunas, B. Ludwig, S. Peer, and F. Ricci. Context [13] H. S. Friedman and S. Booth-Kewley. Personality, type a relevance assessment and exploitation in mobile behavior, and coronary heart disease: the role of emotional recommender systems. Personal and Ubiquitous Computing, expression. Journal of Personality and Social Psychology, 16(5):507–526, 2012. 53(4):783, 1987. [5] L. Baltrunas, B. Ludwig, and F. Ricci. Matrix factorization [14] T. Gilovich, D. Griffin, and D. Kahneman. Heuristics and techniques for context aware recommendation. In biases: The psychology of intuitive judgment. Cambridge Proceedings of the fifth ACM conference on Recommender University Press, 2002. systems, pages 301–304. ACM, 2011. [15] M. Gorgoglione, U. Panniello, and A. Tuzhilin. The effect of preferences. Journal of personality and social psychology, context-aware recommendations on customer purchasing 84(6):1236, 2003. behavior and trust. In Proceedings of the fifth ACM [32] M.-K. Shan, F.-F. Kuo, M.-F. Chiang, and S.-Y. Lee. conference on Recommender systems, pages 85–92. ACM, Emotion-based music recommendation by affinity discovery 2011. from film music. Expert systems with applications, [16] N. Hariri, B. Mobasher, and R. Burke. Context-aware music 36(4):7666–7674, 2009. recommendation based on latenttopic sequential patterns. In [33] Y. Shi, M. Larson, and A. Hanjalic. Mining mood-specific Proceedings of the sixth ACM conference on Recommender movie similarity with matrix factorization for context-aware systems, pages 131–138. ACM, 2012. recommendation. In Proceedings of the workshop on [17] L. Harker and D. Keltner. Expressions of positive emotion in context-aware movie recommendation, pages 34–40. ACM, women’s college yearbook pictures and their relationship to 2010. personality and life outcomes across adulthood. Journal of [34] M. Tkalcic, A. Kosir, and J. Tasic. Affective recommender personality and social psychology, 80(1):112, 2001. systems: the role of emotions in recommender systems. In [18] A. T. Ho, I. L. Menezes, and Y. Tagmouti. E-mrs: Proc. The RecSys 2011 Workshop on Human Decision Emotionbased movie recommender system. In Proceedings Making in Recommender Systems, pages 9–13, 2011. of IADIS e-Commerce Conference. USA: University of [35] M. Tkalcic, M. Kunaver, A. Košir, and J. Tasic. Addressing Washington Both-ell, pages 1–8, 2006. the new user problem with a personality based user similarity [19] R. Hu and P. Pu. Using personality information in measure. In First International Workshop on Decision collaborative filtering for new users. Recommender Systems Making and Recommendation Acceptance Issues in and the Social Web, page 17, 2010. Recommender Systems (DEMRA 2011), page 106. Citeseer, [20] K. Järvelin and J. Kekäläinen. Cumulated gain-based 2011. evaluation of ir techniques. ACM Transactions on [36] J. Vanattenhoven and D. Geerts. Contextual aspects of Information Systems (TOIS), 20(4):422–446, 2002. typical viewing situations: a new perspective for [21] A. KoÅair.˛ Database for contextual personalization. recommending television and video content. Personal and ElektrotehniÅaki ˛ vestnik, 78(5):270–274, 2011. Ubiquitous Computing, 19(5-6):761–779, 2015. [22] F.-F. Kuo, M.-F. Chiang, M.-K. Shan, and S.-Y. Lee. [37] K. Verbert, N. Manouselis, X. Ochoa, M. Wolpers, Emotion-based music recommendation by association H. Drachsler, I. Bosnic, and E. Duval. Context-aware discovery from film music. In Proceedings of the 13th recommender systems for learning: a survey and future annual ACM international conference on Multimedia, pages challenges. IEEE Transactions on Learning Technologies, 507–510. ACM, 2005. 5(4):318–335, 2012. [23] J. Masthoff. The pursuit of satisfaction: affective state in [38] Y. Zheng. A revisit to the identification of contexts in group recommender systems. In User Modeling 2005, pages recommender systems. In Proceedings of the 20th 297–306. Springer, 2005. International Conference on Intelligent User Interfaces [24] R. R. McCrae and O. P. John. An introduction to the Companion, pages 133–136. ACM, 2015. five-factor model and its applications. Journal of personality, [39] Y. Zheng, R. Burke, and B. Mobasher. Differential context 60(2):175–215, 1992. relaxation for context-aware travel recommendation. In [25] Y. Moshfeghi, B. Piwowarski, and J. M. Jose. Handling data International Conference on Electronic Commerce and Web sparsity in collaborative filtering using emotion and semantic Technologies, pages 88–99. Springer, 2012. based features. In Proceedings of the 34th international ACM [40] Y. Zheng, B. Mobasher, and R. Burke. CSLIM: Contextual SIGIR conference on Research and development in SLIM recommendation algorithms. In Proceedings of the 8th Information Retrieval, pages 625–634. ACM, 2011. ACM Conference on Recommender Systems, pages 301–304. [26] A. Odic, M. Tkalcic, J. F. Tasic, and A. Košir. Relevant ACM, 2014. context in a movie recommender system: UsersâĂŹ opinion [41] Y. Zheng, B. Mobasher, and R. Burke. CARSKit: A vs. statistical detection. ACM RecSys, 12, 2012. Java-based context-aware recommendation engine. In [27] C. Ono, Y. Takishima, Y. Motomura, and H. Asoh. Proceedings of the 15th IEEE International Conference on Context-aware preference model based on a study of Data Mining Workshops. IEEE, 2015. difference between real and supposed situation data. UMAP, [42] Y. Zheng, B. Mobasher, and R. Burke. Integrating context 9:102–113, 2009. similarity with sparse linear recommendation model. In [28] U. Panniello, A. Tuzhilin, M. Gorgoglione, C. Palmisano, International Conference on User Modeling, Adaptation, and A. Pedone. Experimental comparison of pre-vs. and Personalization, pages 370–376. Springer, 2015. post-filtering approaches in context-aware recommender [43] Y. Zheng, B. Mobasher, and R. Burke. Similarity-based systems. In Proceedings of the third ACM conference on context-aware recommendation. In International Conference Recommender systems, pages 265–268. ACM, 2009. on Web Information Systems Engineering, pages 431–447. [29] R. W. Picard and R. Picard. Affective computing, volume Springer, 2015. 252. MIT press Cambridge, 1997. [44] Y. Zheng, B. Mobasher, and R. Burke. Emotions in [30] X. Ramirez-Garcia and M. García-Valdez. Post-filtering for a context-aware recommender systems. In Tkalčič, restaurant context-aware recommender system. In Recent B. Decarolis, M. de Gemmis, A. Košir, and A. Odič, editors, Advances on Hybrid Approaches for Designing Intelligent Emotions and Personality in Personalized Services, pages Systems, pages 695–707. Springer, 2014. 311–326. Springer Berlin Heidelberg, 2016. [31] P. J. Rentfrow and S. D. Gosling. The do re mi’s of everyday [45] Y. Zheng, B. Mobasher, and R. D. Burke. The role of life: the structure and personality correlates of music emotions in context-aware recommendation. Decisions@ RecSys, 2013:21–28, 2013.