=Paper=
{{Paper
|id=Vol-1680/paper1
|storemode=property
|title=Adapt to Emotional Reactions in Context-aware Personalization
|pdfUrl=https://ceur-ws.org/Vol-1680/paper1.pdf
|volume=Vol-1680
|authors=Yong Zheng
|dblpUrl=https://dblp.org/rec/conf/recsys/Zheng16
}}
==Adapt to Emotional Reactions in Context-aware Personalization==
https://ceur-ws.org/Vol-1680/paper1.pdf
Adapt to Emotional Reactions In Context-aware
Personalization
Yong Zheng
Department of Information Technology and Management
School of Applied Technology
Illinois Institute of Technology
Chicago, Illinois, USA
yzheng66@iit.edu
ABSTRACT attributes as the observed contexts which may change when the user
Context-aware recommender systems (CARS) have been devel- performs the same activity repeatedly [38]. For example, the time
oped to adapt to users’ preferences in different contextual situa- and location may change every time when a user tries to watch a
tions. Users’ emotions have been demonstrated as one of effective movie. The season or trip type may change when a user is going to
context information in recommender systems. However, there are reserve a hotel. In addition to these factors, users’ emotional states
no work exploring the effect of emotional reactions (or expressions) are one of these dynamic variables. And emotions may change
in the recommendation process. In this paper, we assume that users anytime in the process of user interactions with the items or the
may give similar ratings even if they present different emotional applications. These emotional information have been demonstrated
reactions or expressions on the movies. We further model the traits as effective and influential context in previous research [45, 44].
of emotional reactions and incorporate them into context-aware Emotional reactions or expressions are highly correlated with
matrix factorization as regularization terms. Our experimental the traits of user personalities. Personality accounts for the most
results based on the LDOS-CoMoDa movie data set validate our important ways in which individuals differ in their enduring emo-
assumptions and prove that it is useful to take emotional reactions tional, interpersonal, experimental, attitudinal and motivational
into consideration in context-aware recommendations. styles [24]. In the domain of recommender systems, personality
can be viewed as a user profile, which may be context-independent
and domain-independent. Both emotional information [18, 11,
Keywords 34, 45] and user personality [31, 19, 35] have been successfully
context, recommendation, emotion, emotional reactions incorporated into recommender systems by existing research.
Our previous research [45] has successfully utilized emotional
variables as contexts in recommender systems to improve recom-
Categories and Subject Descriptors mendation performance. Unfortunately, as far as we know, there
H.3.3 [Information Search and Retrieval]: Information filtering, are no research on exploring the effect of emotional reactions
Retrieval models; H.1.2 [Models and Principles]: User/Machine or expressions. We believe that users’ emotional reactions or
Systems - human information processing expressions are also useful to model users’ preferences or rating
behaviors in real practice. For example, two different users may
1. INTRODUCTION AND BACKGROUND give high ratings on a same tragedy drama movie. One of them
indicated his or her emotional state as "happy" when finishing the
Recommender systems (RS) are an effective way in alleviating movie, because this user thought it was a really good movie. By
information overload by tailoring recommendations to users’ per- contrast, another user may express his or her feeling as "sad" since
sonal preferences. Context-aware recommender systems (CARS) the user was impressed or moved by the tragedy movie. As a result,
take contextual factors (such as time, location, companion, occa- the two users have same rating behaviors on the movie but with
sion, etc) into account in modeling user profiles and in generating different emotional reactions or expressions. One of the potential
recommendations. For example, users’ choice on movies may be reasons is that different user personalities may result in different
very different if the user is going to watch the movie with children ways or habits for users to express their emotions.
rather than with his or her partner. Therefore, the users’ rating profiles associated with different (or
Context, is usually defined as, "any information that can be even opposed) emotional reactions therefore could be useful to
used to characterize the situation of an entity. An entity is assist recommendations. In this paper, we propose to incorporate
a person, place, or object that is considered relevant to the emotional reactions (or expressions) as regularization terms in the
interaction between a user and an application, including the user context-aware matrix factorization approach, and further explore
and applications themselves [12]". In CARS, we view the dynamic its effect on the performance of context-aware recommendations.
The following sections are organized as follows: Section 2
introduces related work, including the background of context-
aware recommendation, and the role of emotions and personality in
recommender systems. Section 3 gives the preliminary description
of essential information, such as the LDOS-CoMoDa movie data
which contains emotional variables, and the introduction about
the CAMF technique. Section 4 discusses our methodology
EMPIRE 2016, September 16, 2016, Boston, MA, USA.
Copyright held by the author(s). that incorporates the emotional reactions as regularization terms
into the CAMF approach. Section 5 describes our experimental to explore the effect of emotional reactions in the context-aware
results and discussions, followed by Section 6 which concludes our recommendations.
findings and discusses our future work.
3. PRELIMINARY
2. RELATED WORK To further discuss the topics in the context-aware recommendation,
One of the goals in the recommender systems (RS) is to assist it is necessary to introduce some terminologies:
users’ decision making by providing a list of recommendations.
Due to the fact that users’ choice usually varies from time to time Table 1: Sample of a Context-aware Movie Rating Data Set
and from context to context, context-aware recommender systems
(CARS) [2, 1] are promoted and developed to adapt to users’ User Movie Rating Time Location Companion
preferences in different contextual situations. U1 T1 5 Weekday Home Kids
In rating-based RS applications, such as movie or book ratings, U1 T1 3 Weekend Cinema Family
the standard formulation of the recommendation problem begins U2 T2 3 Weekday Cinema Partner
with a two dimensional matrix of ratings, organized by user and U2 T3 4 Weekday Home Kids
item: Users × Items → Ratings. The key insight of CARS is that U3 T4 2 Weekend Home Partner
users’ preferences on items may be also a function of the context in
which those items are encountered. Incorporating contexts requires Table 1 shows an example of context-aware movie data which
that we estimate user preferences using a multidimensional rating contains five rating profiles given by three users on four movies
function, Users × Items × Contexts → Ratings [1]. in different contextual situations. In our discussions, we will use
In the past decade, several context-aware recommendation algo- the term contextual dimension to denote the contextual variable,
rithms have been developed. By additionally incorporating context such as "Location", "Time" and "Companion". The term contextual
information, these algorithms have been demonstrated to be useful condition refers to a specific value in a contextual dimension,
to improve recommendation performance in numerous domains, e.g. "Home" and "Cinema" are two contextual conditions for the
such as e-commerce [28, 15], movies [33, 26, 10], music [3, 16], dimension "Location". Context or contextual situation therefore
restaurants [30, 27], travels [39, 8], educational learning [37], refers to a combination of contextual conditions, e.g., {Weekday,
mobile applications [4, 6], and so forth. The context variables Home, Kids}.
adopted in those applications are domain-specific ones. And the Next, we introduce the LDOS-CoMoDa movie data 1 which is
most widely used context information are the time of the day, the a data set with multiple contextual dimensions including several
day of the week, and location information which can be easily emotional variables. We also introduce context-aware matrix
captured from ubiquitous environment, such as Web logs, mobile factorization which is a popular algorithm in CARS and we use
devices, sensors. it as a base algorithm in this paper.
It is well known that human decision making is subject to both
rational and emotional influences [14]. The field of affective 3.1 LDOS-CoMoDa Data Set
computing takes this fact as basic to the design of computing In the domain of context-aware recommendation, there are very
systems [29]. The role of emotions in recommender systems limited number of data sets available for public research, not to
was recognized by the research community as early as 2005 [23], mention the data that contains emotional variables. The LDOS-
giving rise to research in emotion-based movie recommender CoMoDa data set [21] introduced below is one of the data sets that
systems [18] and the impact of emotions in group recommender was collected from user surveys, and can be used for this type of
systems [23, 11]. This results in the highlight of research on research in this paper. The data has 2291 ratings (rating scale is
affective recommender systems [34] which have been proved to be 1 to 5) given by 121 users on 1232 items within 12 contextual
useful on recommendation performance in several domains, such dimensions. The description of the contextual dimensions and
as music [22, 32, 9] and movies [7, 25, 18]. conditions can be described by Table 2.
Emotional states, accordingly, are also viewed and used as
contexts in recommender systems. Shi et al. [33] mined the mood Table 2: List of Context Information in the LDOS-CoMoDa Data
similarity to assist context-aware movie recommendation. Odic,
et al. [26] identified the significant contributions by emotional Dimension Contextual Conditions
variables compared with other contextual factors in the LDOS- Time Morning, Afternoon, Evening, Night
CoMoDa movie rating data. Mood information can also be used Daytype Working day, Weekend, Holiday
for television and video content recommendation [36]. Baltrunas, Season Spring, Summer, Autumn, Winter
Location Home, Public place, Friend’s house
et al. [3] adopted mood as context to assist context-aware music Weather Sunny / clear, Rainy, Stormy, Snowy, Cloudy
recommendation. The role of emotions in context-aware recom- Companion Alone, Partner, Friends, Colleagues, Parents, Public, Family
mendation is summarized in [45, 44] which helps additionally endEmo Sad, Happy, Scared, Surprised, Angry, Disgusted, Neutral
discover insights about why and where emotional states play an domEmo Sad, Happy, Scared, Surprised, Angry, Disgusted, Neutral
important role in the recommendation process. Mood Positive, Neutral, Negative
Emotional states are usually dynamic and may change from time Physical Healthy, Ill
Decision Movie choices by themselves or users were given a movie
to time. Based on the introduction about the affective recommender Interaction First interaction with a movie, Nth interaction with a movie
systems [34], the emotional information in three stages may be
useful: entry stage (i.e., before the activity), consumption stage
Among these 12 contextual dimensions, there are three ones
(i.e., during the activity) and exit stage (i.e. after the activity).
that can be considered emotional dimensions: endEmo, domEmo
In this case, emotional reactions can be captured across these
and mood. "endEmo" is the emotional state experienced at the
three stages. As introduced previously, users may present different
end of the movie (i.e., emotion in the exit stage). "domEmo" is
emotional reactions, but actually they leave the same or similar
1
ratings on the items. In this paper, we make the first attempt LDOS-CoMoDa data set, http://www.ldos.si/comoda.html
the emotional state experienced the most during watching (i.e.,
emotion in the consumption stage). "mood" is the emotion of
N
the user during that part of the day when the user watched the X 1 2 λ X
2 2 2 2
min err + ( B + bi + ||pu || + ||qi || )
movie (i.e., emotion in the entry stage). "EndEmo" and "domEmo" B∗ ,b∗ ,p∗ ,q∗
r∈R
2 2 j=1 u,cj
contain the same seven conditions: Sad, Happy, Scared, Surprised, (3)
Angry, Disgusted, Neutral, while "mood" only has three simple Afterwards, the algorithm is able to learn the corresponding
conditions: Positive, Neutral, Negative. parameters by minimizing the squared errors in prediction. The
Context selection is usually performed before we apply any loss function as shown in Equation 3 is a composition of squared
context-aware recommendation algorithms. We’d like to retain error and regularization terms which are used to alleviate the
the most influential context dimensions, since irrelevant ones may overfitting problems, where ruic1 c2 ...cN is the real and known
introduce noises in the data and further hamper the recommenda- rating given by user u on item i in context c1 c2 ..cN , and λ is the
tion accuracy. Based on the statistical selection method introduced regularization rate used in the optimization process. By stochastic
in [26], we only use 7 out of the 12 contextual dimensions in our gradient descent, we are able to learn the parameters iteratively and
experiments: time, daytype, location, companion and the three finally achieve the best performing CAMF_CU model.
emotional variables. CAMF is an effective algorithm and it is able to alleviate the data
The three emotional variables (i.e., mood, domEmo and en- sparsity to some extent. We choose CAMF_CU because we are
dEmo) describe users’ affective states during the user interactions going to explore the correlation between users and their emotional
with the movies in terms of three stages respectively: entry stage, reactions, which requires a user-specific context-dependent model.
consumption stage and exit stage as introduced in [34]. In other The same thing can also happen to other algorithms which explore
words, mood can be viewed as current context before the user intersections or the dependency between users and contexts, such
starts watching the movie. By contrast, domEmo and endEmo can as the CSLIM_CU approach [40].
indicate future emotional states during the user’s interactions with In the next section, we will introduce how to incorporate the
the activity of movie watching. These future status can also be emotional reactions as regularization terms to CAMF_CU.
viewed as contexts too if we interpret them as user intents. For
example, a user is feeling sad now, and he or she wants to select
a movie to watch in order to be happy. In this example, "sad" is 4. METHODOLOGY
the current user mood, and "happy" can be viewed as user’s future In this section, we introduce our methodology of how to incorpo-
emotional state, such as in the domEmo or endEmo. rate emotional reactions into context-aware recommender systems.
4.1 Problem Statement
3.2 Context-aware Matrix Factorization Recall that we assume that the different emotional reactions or
One of the most popular context-aware recommendation algo- expressions can be used to model users’ rating behaviors. For
0123ÿ0123ÿ0125
rithms is the one built upon matrix factorization, namely, the example, assume two users gave a high rating on a same tragedy
context-aware matrix factorization (CAMF) approach [5]. There drama movie. One of them indicated his or her emotional state
are different variants of CAMF, here we introduce the CAMF_CU as "happy" when finishing the movie, because this user thought
approach which incorporate a user-personalized contextual rating it was a really good movie. But another user may express his
bias into matrix factorization. More specifically, the rating predic- or her feeling as "sad" since it is a tragedy movie. The same
6789 ÿ9
tion function by CAMF_CU can be described by Equation 1. thing may also happen to the domEmo in addition to the endEmo.
The emotional reactions or expressions in this paper, refer to the
ÿ ÿ
N
X different values in the dimension domEmo and/or endEmo in the
r̂uic1 c2 ...cN = µ + Bu,cj + bi + pTu qi (1) LDOS-CoMoDa data.
ÿ ÿ!
j=1 Figure 1 presents the distribution of rating counts in each
Assume there are totally N contextual dimensions. c1 c2 ..cN is emotional state. Note that "Unknown" indicates the missing value
6"#9# ÿ9
used to denote a contextual situation, where c1 indicates the value in the LDOS-CoMoDa. We can observe that Neutral and Happy
of contextual condition in the 1st context dimension. r̂uic1 c2 ...cN are the most two common emotional expressions in both domEmo
$%ÿ
therefore represents the predicted rating for user u on item i in and endEmo.
the situation c1 c2 ..cN . The prediction function is composed of 1000
&''2ÿ 800
four components: the global mean rating µ, item rating bias bi ,
N
700
($')ÿ
P
the aggregated contextual rating bias Bu,cj , and user-item
600
9:;<ÿ#=ÿ789 >
j=1
interaction represented by the dot product of a user vector and item 500
()ÿ 400
vector, pTu qi . pu is the user vector represented by a set of latent
factors, and qi is the item vector represented by the same set of 00
* $)ÿ
factors. pu can tell how much the user u likes those latent factors, 300
while qi indicates how the item i obtains these factors. Therefore, 200
(+)ÿ
the dot product function is used to estimate how much the user will 100
like this item. 0
$% &''2 ($') () * $) (+) , 2 -./
, 2ÿ
The term Bu,cj is the estimated contextual rating bias for user u
in context condition cj . It is used to denote how user u’s rating is )?@? )@?
-./ÿ
deviated in each contextual condition.
Figure 1: Distribution of Rating Counts in Each Emotional State
err = ruic1 c2 ...cN − r̂uic1 c2 ...cN (2)
0123ÿ0123ÿ0125
67898ÿ9899
ÿ!!ÿ" #$%&' 0 ÿ!!ÿ" #$%&'
(
)
9ÿ89 89ÿ89 *++8,ÿÿ-./. *++8,ÿÿ-/.
2 ÿ34'$#$56ÿ" #$%&' 8 ÿ96& #$56ÿ" #$%&'
))
7
1 7)
89ÿÿ-./. 89ÿÿ-/. 9ÿÿ-./. 9ÿÿ-/.
Figure 2: Distribution of Unusual Emotional Reactions in the LDOS-CoMoDa Data
Furthermore, we’d like to learn the unusual case to see whether problem can be summarized as how to incorporate these emotional
users present different emotional reactions in this data. An unusual reactions into existing recommendation algorithms. More specif-
case could be two situations: 1). a user leaves a negative rating, but ically, we want to explore the approach to incorporate them into
expresses positive emotional states in either domEmo or endEmo; the CAMF approach. There are three questions we are particularly
2). a user gives a positive rating while he finally indicates a negative interested in:
emotion in either domEmo or endEmo.
To explore these unusual cases, we need to define which ratings • How to fuse this emotional reactions into CAMF?
and which emotions are positive or negative. In our experiments,
we simply view a rating as a positive one if the rating is no less • Does it work by providing improvements?
than 4; otherwise, the rating is negative. In terms of the emotional
• Which emotional reaction is more effective? The reactions
states, we only consider "Happy" and "Surprised" as positive ones,
based on domEmo or endEmo?
while other emotional states are negative.
A simple statistics about unusual cases in this data can be
depicted by Figure 2. First of all, 66.5% of the ratings are positive
4.2 Regularization by Emotional Reactions
ones as shown in subfigure a). Based on the subfigure b), we First of all, how the user reacts on the movies in terms of emotional
can observe that 36.9% of all the rating records are unusual cases status is dependent with what type of movies the user is watching.
(i.e., the two situations mentioned above) based on the domEmo In this case, we additionally use movie genre information in the
variable, while it is 33.4% in the endEmo variable. This may tell LDOS-CoMoDa data and aggregated users’ ratings for each movie
that domEmo could be more effective and useful than endEmo in type. A sample of the aggregated data can be shown in Table 3.
modeling users’ emotional reactions.
The subfigure c) and d) further describe the two unusual situa- Table 3: An Example of Aggregated Rating Matrix
tions among the positive and negative ratings respectively. In the
piece of profiles with positive ratings, 43.4% of them are associated User Genre Rating Time domEmo endEmo
with negative emotions in domEmo – many more than the cases in U1 Action 5 Weekday Sad Happy
endEmo. It is not surprising, since the theme or the genre of the U1 Drama 3 Weekend Sad Sad
movie will affect user’s dominating emotions during the process of U2 Cartoon 3 Weekday Happy Angry
movie watching. For example, users may feel horrible or scared U2 Drama 3 Weekday Angry Happy
when watching a horrible movie, but finally leave a positive rating U3 Action 2 Weekend Sad Sad
since it is a good movie. On the other hand, in terms of the records
with negative ratings, there are no significant differences for the In Table 3, we replace the column of item by movie genre
unusual cases between domEmo (24%) and endEmo (25%) based to construct a new rating matrix. We will use the same 7
on the observations subfigure d). Recall that, there are many more contextual dimensions introduced previously. Note that in the
positive ratings than the negative ones in this data. Therefore, it LDOS-CoMoDa data we do not know what the movie genre is,
seems that users may express more unusual emotional reactions in since the genre was encoded as numbers in this data.
domEmo rather than in endEmo. We suspect that the emotional Afterwards, we can fuse an emotional dimension (either endEmo
reactions in domEmo may leave more influential impact on our or domEmo) into the user dimension to create a two-dimensional
proposed recommendation models. rating matrix. Let’s take the domEmo for example, the converted
The underlying assumption in our proposed approach is that rating matrix can be described by Table 4.
user’s emotional reactions or expressions on the future emotional Specifically, we fuse the values in domEmo into the user column
states (e.g., domEmo and/or endEmo) can be used to improve rec- to create new users. The new user is represented by a combination
ommendations, since they may indicate similar user tastes even if of original user ID and value in the domEmo, and we name those
the emotional reactions are different or even opposed. The research new users as emotional users. Meanwhile, we eliminate the other
Table 4: Converted Two-Dimensional Rating Matrix the emotional state in domEmo, since it is not necessary to be the
same value as cm . But they should be the contextual condition in
User, domEmo Genre Rating the same dimension (i.e., the mth dimension).
U1, Sad Action 5 As mentioned previously, more similar two emotion users are,
U1, Sad Drama 3 their ratings on the items (with same genre) should be similar.
U2, Happy Cartoon 3 In our CAMF_CU model, it can be derived that user’s contextual
U2, Angry Drama 3 rating deviations in this emotional variable (i.e., the mth contextual
U3, Sad Action 2 variable) should be similar. Namely, Bu,cm and Bv,cm+ should be
very close. We add the squared difference of these two deviations
(e.g., Equation 5) as the regularization term in Equation 4.
contextual dimensions from the rating matrix. In this case, we can Additionally, how close the two contextual rating deviations are
build a matrix factorization model based on this converted two- should be dependent with the similarity of two emotional users.
dimensional rating matrix. And then we are able to calculate the In this case, the regularization term is weighted by the similarity
similarity between emotional users based on the cosine similarity between two emotional users. We name this term as "emotional
of each two vectors which represent emotional users. For example, regularization term" in this paper.
we can measure how similar the "U1, Sad" to "U2, Angry" based Recall that our assumption is that the emotional users should
on their co-ratings on the movies with the same genre information. be similar because two difference users have similar ratings even
Theoretically, we can use the item information (e.g., item if their emotional reactions are different. It can also tell that the
ID) instead of the movie genre in the rating matrix, but it will two users actually share something in common, so we assume
increase data sparsity. We use movie genre information only for there should also be a similarity between two users to some
two reasons: On one hand, using movie genre is based on our extent. Therefore, we are able to additionally incorporate a
assumptions that users’ different emotional reactions depend on "user regularization term" to build a finer-grained recommendation
the movie genre and user’s emotional reactions, for example, user model, where the loss function can be shown in Equation 6. Again,
may express as happy or sad on a tragedy movie. On the other the user regularization is also weighted by the similarity between
hand, it is able to alleviate the rating sparsity in the converted two- two emotional users.
dimensional rating matrix so that we can obtain more reliable user
similarities. We have tried to use item ID, but emotional users
N
1 err 2 + λ 2 + b2 2 2
P
have very few co-ratings on the items, which results in worse X 2 2
( Bu,c
j i + ||pu || + ||qi || )
min j=1
β
recommendation performance compared with that when we use B∗ ,b∗ ,p∗ ,q∗
r∈R + 2
P
Sim((v, cm+ ), (u, cm )) × (reg_user + reg_emo)
v,cm+ ∈K
genre information only. Note that we use domEmo as an example (6)
in Table 4, while we can also have the same process based on the
variable endEmo. reg_user = ||pu − pv ||
2
(7)
In short, the emotional users should be similar if they have
Based on those two different loss functions, we are able to build
similar ratings on the movies with same genre information, even
two new CAMF approaches by incorporating the emotional reac-
if the original users have different emotional reactions on domEmo
tions as the regularization terms. We can learn the corresponding
or endEmo. For example, the ratings given by "U1, Sad" and "U2,
parameters based on the gradient decent accordingly. Note that the
Angry" are all 3-star on the drama movies shown in the Table 4.
performance of the models may also depend on the number of K-
Therefore, U1 with dominating emotion as "Sad" may share similar
nearest neighbors used in the algorithm. In our experiments, we set
user tastes with U2 with dominating emotion as "Angry" to some
different values to explore the best options in these parameters.
extent.
Accordingly, we are able to create a regularization term based
on the similarity of contextual users. The new loss function can be 5. EXPERIMENTS
shown as Equation 4, where β is the regularization rate for the new In this section, we introduce our evaluation settings and experimen-
regularization terms. tal results, as well as our findings.
N
5.1 Evaluation Protocols
1 err 2 + λ 2 + b2 2 2
P
X 2 2
( Bu,c
j i + ||pu || + ||qi || ) We employ a 5-folds cross-validation on the LDOS-CoMoDa data
min j=1
(4)
β
B∗ ,b∗ ,p∗ ,q∗
r∈R + 2
P
Sim((v, cm+ ), (u, cm )) × reg_emo
set. Namely, we split the rating profiles into 5 folds and perform
v,cm+ ∈K
5 rounds evaluations. For each round, one of the fold will be used
2
as testing set, and the other 4 folds of data will be used as training
reg_emo = (Bu,cm − Bv,c ) (5)
m+
data. We build our recommendation models based on the training
We will use the same function shown in Equation 1. In addition, set and evaluate the results according to the ground truth inferred
we incorporate a new regularization term in Equation 4 compared from the testing set.
with the loss function described by Equation 3. We use CAMF_CU approach as baseline, and compete its
More specifically, we use m to denote the index of an emotional recommendation performance with the CAMF_CU models with
variable (i.e., either domEmo or endEmo). Take domEmo for different regularization terms. We use the CAMF_CU approach
example, m indicates the position of domEmo in the list of implemented in the open-source toolkit, CARSKit [41], to perform
contextual dimensions, thus cm is used to express user’s emotional the evaluations.
state in domEmo. According, "u, cm " is the emotional user More specifically, we evaluate the recommendation performance
(introduced as Table 4), and we use K to denote the top-K nearest based on the rating prediction and top-10 recommendation tasks.
neighbor of emotional user "u, cm " based on the user similarity In the rating prediction task, we use mean absolute error (MAE)
calculated based on the matrix factorization model built upon the as evaluation metric. We also further examine the statistical
converted two-dimensional rating matrix. Namely, "v, cm+ " is one difference of MAE among different algorithms based on paired t-
of the identified top-K nearest neighbors. We use cm+ to denote test at a 95% confidence level. In the top-10 recommendation, We
0.73
0.72 Precision NDCG
0.012 0.012
0.71
0.7 0.01 0.01
MAE
0.69 0.008 0.008
Precision
NDCG
0.68 0.006 0.006
0.67 0.004 0.004
0.66 0.002 0.002
0.65 0 0
CAMF_CU domEmo_B domEmo_B,u endEmo_B endEmo_B,u CAMF_CU domEmo_B domEmo_B,u endEmo_B endEmo_B,u
(a) Results on MAE (b) Results on Top-10 Recommendation
Figure 3: Experimental Results on the Rating Prediction and Top-10 Recommendation Tasks
adopt precision as the relevance metric and Normalized Discounted endEmo_B, u outperforms endEmo_B (30.1% improvement on
Cumulative Gain (NDCG) [20] as the ranking metric. More precision, and 18.5% on NDCG).
specifically, precision is calculated as the ratio of relevant items As mentioned before, the number of selected neighbors in
selected to the number of items recommended (i.e., 10 in our our models may impact the recommendation performance. We
experiment). NDCG is a measure from information retrieval, where present the impact by the number of neighbors in the finer-grained
positions are discounted logarithmically. CAMF_CU approaches with two regularization terms, as shown
by the Figure 4. Simply, we vary the number of neighbors from 10
to 80 with an increment of 10 on each step. The best number of
5.2 Experimental Results neighbors should be around 40 to 50 in this data set. It is essential
First of all, we present our results based on the rating prediction to examine different number of neighbors to find out the optimal
task in Figure 3(a). We use CAMF_CU to denote the original selection for each recommendation model.
approach without emotional or user regularization terms. Our Finally, the experimental results help us identify that the domEmo
approaches introduced in this paper are built upon CAMF_CU is more useful and effective to be adopted than using endEmo. This
approach and they can be generated based on either domEmo or finding is consistent with our previous analysis on the unusual cases
endEmo. We evaluate the performances by them individually. shown in Figure 2. It makes sense since the emotional status during
We use "domEmo_B" to represent the model using domEmo for the process of movie watching may be very different than their
emotional regularization, i.e., cm denotes the emotional state in emotions at the end. For example, a user may feel horrible if he
domEmo in Equations 4. By contrast, "domEmo_B, u" is used to or she is watching an adventure movie, but finally he or she might
denote the finer-grained model described in Equation 6 which con- feel happy since it is a good movie.
tains both emotional and user regularization terms. Accordingly,
"endEmo_B" and "endEmo_B, u" are the two recommendation 5.3 Discussions
models by using endEmo to generate the regularization terms. Why emotional reactions or expressions can be reused to improve
Based on the results shown in Figure 3(a), our proposed ap- the recommendation performance? As we mentioned before, one
proaches only using the emotional regularization term can help of the potential reasons is that the different emotional reactions
obtain lower MAE. All of these improvements are statistically are caused by the traits in different user personalities – users may
significant based on the paired t-test. When we try to use both express their emotional states or reactions in different ways. It
emotional and user regularization terms, it is able to further has been well studied that the emotional expression has strong
improve prediction accuracies. However, the improvement by correlations with user personality, especially in the areas of psy-
endEmo_B,u fails the paired t-test compared with the endEmo_B chology and social science. For example, the correlation between
approach. The best performing model in the rating prediction task emotional expression and personality can be used to assist health
is domEmo_B,u, where we apply emotional and user regularization care [13]. Harker, et al. [17] found that individual differences in
terms at the same time, and these regularization terms are generated positive emotional express were linked to personality stability and
based on the emotional reactions by domEmo. development across adulthood. However, there are no applications
We show the top-10 recommendation results based on precision of using personality inferred from emotional reactions or expres-
and NDCG in Figure 3(b). The bars present results based on sions to further serve real-world applications, such as recommender
precision at top-10 recommendation, the curve tells the results systems. In this paper, we make our attempts to explore the impacts
in NDCG. We can observe similar patterns shown in the rating of emotional reactions or expressions in the recommender systems,
prediction task: first, we see that the CAMF_CU models with our especially in the context-aware personalization.
regularization terms are able to outperform the original CAMF_CU
approach in both precision and NDCG. This finding confirms that
incorporating emotional regularization terms inferred from users’ 6. CONCLUSIONS AND FUTURE WORK
emotional reactions is helpful to improve performance of context- In this paper, we believe that users may place similar ratings even
aware recommendation. if they may have different emotional reactions or expressions. We
Furthermore, we can observe the finer-grained model with addi- propose to incorporate the corresponding regularization terms in
tional user regularization term contributes to obtain more improve- the CAMF_CU approach to assist context-aware recommendation.
ments. For example, domEmo_B,u works better than domEmo_B Our findings based on the experimental results over the LDOS-
(19.6% improvement on precision, and 18.2% on NDCG), and CoMoDa movie data demonstrate that modeling user’s emotional
Results by domEmo_B Results by domEmo_B,u
Precision NDCG Precision NDCG
0.0025 0.01 0.0025 0.012
0.002 0.008 0.002 0.01
0.008
Precision
Precision
0.0015 0.006 0.0015
NDCG
NDCG
0.006
0.001 0.004 0.001
0.004
0.0005 0.002 0.0005 0.002
0 0 0 0
10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80
the number of neighbors the number of neighbors
Results by endEmo_B Results by endEmo_B,u
Precision NDCG Precision NDCG
0.0015 0.008 0.002 0.01
0.006 0.0015 0.008
0.001
Precision
Precision
0.006
NDCG
NDCG
0.004 0.001
0.004
0.0005
0.002 0.0005 0.002
0 0 0 0
10 20 30 40 50 60 70 80 10 20 30 40 50 60 70 80
the number of neighbors the number of neighbors
Figure 4: Impact by the Number of Neighbors
reactions is helpful to improve recommendation performance. The [6] M. J. Barranco, J. M. Noguera, J. Castro, and L. Martínez. A
results also reveal that domEmo is better than endEmo to generate context-aware mobile recommender system based on
the regularization terms in this data set. And the finer-grained location and trajectory. In Management intelligent systems,
model by additionally incorporating user regularization is able to pages 153–162. Springer, 2012.
offer further improvements. [7] S. Benini, L. Canini, and R. Leonardi. A connotative space
One of our future work is to incorporate these regularization for supporting movie affective recommendation. Multimedia,
terms based on different emotional reactions to more context-aware IEEE Transactions on, 13(6):1356–1370, 2011.
recommendation models. It is interesting to examine the similar [8] M. Braunhofer, M. Elahi, and F. Ricci. Sts: A context-aware
approach in the similarity-based context-aware recommendation mobile recommender system for places of interest. In UMAP
algorithms [43, 42] so that we can learn the similarities of not only Workshops. Citeseer, 2014.
the emotional users but also the emotion themsevles. We will also [9] R. Cai, C. Zhang, C. Wang, L. Zhang, and W.-Y. Ma.
try to explore the effect of emotional reactions in other applications Musicsense: contextual music recommendation using
(such as music) rather than the movie domain. emotional allocation modeling. In Proceedings of the 15th
international conference on Multimedia, pages 553–556.
7. REFERENCES ACM, 2007.
[1] G. Adomavicius, B. Mobasher, F. Ricci, and A. Tuzhilin. [10] P. G. Campos, I. Fernández-Tobías, I. Cantador, and F. Díez.
Context-aware recommender systems. AI Magazine, Context-aware movie recommendations: an empirical
32(3):67–80, 2011. comparison of pre-filtering, post-filtering and contextual
[2] G. Adomavicius, R. Sankaranarayanan, S. Sen, and modeling approaches. In E-Commerce and web technologies,
A. Tuzhilin. Incorporating contextual information in pages 137–149. Springer, 2013.
recommender systems using a multidimensional approach. [11] Y. Chen and P. Pu. Cofeel: Using emotions to enhance social
ACM Transactions on Information Systems (TOIS), interaction in group recommender systems. In Alpine
23(1):103–145, 2005. Rendez-Vous (ARV) 2013 Workshop on Tools and Technology
[3] L. Baltrunas, M. Kaminskas, B. Ludwig, O. Moling, F. Ricci, for Emotion-Awareness in Computer Mediated
A. Aydin, K.-H. Lüke, and R. Schwaiger. Incarmusic: Collaboration and Learning, 2013.
Context-aware music recommendations in a car. In EC-Web, [12] A. K. Dey. Understanding and using context. Personal and
volume 11, pages 89–100. Springer, 2011. ubiquitous computing, 5(1):4–7, 2001.
[4] L. Baltrunas, B. Ludwig, S. Peer, and F. Ricci. Context [13] H. S. Friedman and S. Booth-Kewley. Personality, type a
relevance assessment and exploitation in mobile behavior, and coronary heart disease: the role of emotional
recommender systems. Personal and Ubiquitous Computing, expression. Journal of Personality and Social Psychology,
16(5):507–526, 2012. 53(4):783, 1987.
[5] L. Baltrunas, B. Ludwig, and F. Ricci. Matrix factorization [14] T. Gilovich, D. Griffin, and D. Kahneman. Heuristics and
techniques for context aware recommendation. In biases: The psychology of intuitive judgment. Cambridge
Proceedings of the fifth ACM conference on Recommender University Press, 2002.
systems, pages 301–304. ACM, 2011.
[15] M. Gorgoglione, U. Panniello, and A. Tuzhilin. The effect of preferences. Journal of personality and social psychology,
context-aware recommendations on customer purchasing 84(6):1236, 2003.
behavior and trust. In Proceedings of the fifth ACM [32] M.-K. Shan, F.-F. Kuo, M.-F. Chiang, and S.-Y. Lee.
conference on Recommender systems, pages 85–92. ACM, Emotion-based music recommendation by affinity discovery
2011. from film music. Expert systems with applications,
[16] N. Hariri, B. Mobasher, and R. Burke. Context-aware music 36(4):7666–7674, 2009.
recommendation based on latenttopic sequential patterns. In [33] Y. Shi, M. Larson, and A. Hanjalic. Mining mood-specific
Proceedings of the sixth ACM conference on Recommender movie similarity with matrix factorization for context-aware
systems, pages 131–138. ACM, 2012. recommendation. In Proceedings of the workshop on
[17] L. Harker and D. Keltner. Expressions of positive emotion in context-aware movie recommendation, pages 34–40. ACM,
women’s college yearbook pictures and their relationship to 2010.
personality and life outcomes across adulthood. Journal of [34] M. Tkalcic, A. Kosir, and J. Tasic. Affective recommender
personality and social psychology, 80(1):112, 2001. systems: the role of emotions in recommender systems. In
[18] A. T. Ho, I. L. Menezes, and Y. Tagmouti. E-mrs: Proc. The RecSys 2011 Workshop on Human Decision
Emotionbased movie recommender system. In Proceedings Making in Recommender Systems, pages 9–13, 2011.
of IADIS e-Commerce Conference. USA: University of [35] M. Tkalcic, M. Kunaver, A. Košir, and J. Tasic. Addressing
Washington Both-ell, pages 1–8, 2006. the new user problem with a personality based user similarity
[19] R. Hu and P. Pu. Using personality information in measure. In First International Workshop on Decision
collaborative filtering for new users. Recommender Systems Making and Recommendation Acceptance Issues in
and the Social Web, page 17, 2010. Recommender Systems (DEMRA 2011), page 106. Citeseer,
[20] K. Järvelin and J. Kekäläinen. Cumulated gain-based 2011.
evaluation of ir techniques. ACM Transactions on [36] J. Vanattenhoven and D. Geerts. Contextual aspects of
Information Systems (TOIS), 20(4):422–446, 2002. typical viewing situations: a new perspective for
[21] A. KoÅair.˛ Database for contextual personalization. recommending television and video content. Personal and
ElektrotehniÅaki ˛ vestnik, 78(5):270–274, 2011. Ubiquitous Computing, 19(5-6):761–779, 2015.
[22] F.-F. Kuo, M.-F. Chiang, M.-K. Shan, and S.-Y. Lee. [37] K. Verbert, N. Manouselis, X. Ochoa, M. Wolpers,
Emotion-based music recommendation by association H. Drachsler, I. Bosnic, and E. Duval. Context-aware
discovery from film music. In Proceedings of the 13th recommender systems for learning: a survey and future
annual ACM international conference on Multimedia, pages challenges. IEEE Transactions on Learning Technologies,
507–510. ACM, 2005. 5(4):318–335, 2012.
[23] J. Masthoff. The pursuit of satisfaction: affective state in [38] Y. Zheng. A revisit to the identification of contexts in
group recommender systems. In User Modeling 2005, pages recommender systems. In Proceedings of the 20th
297–306. Springer, 2005. International Conference on Intelligent User Interfaces
[24] R. R. McCrae and O. P. John. An introduction to the Companion, pages 133–136. ACM, 2015.
five-factor model and its applications. Journal of personality, [39] Y. Zheng, R. Burke, and B. Mobasher. Differential context
60(2):175–215, 1992. relaxation for context-aware travel recommendation. In
[25] Y. Moshfeghi, B. Piwowarski, and J. M. Jose. Handling data International Conference on Electronic Commerce and Web
sparsity in collaborative filtering using emotion and semantic Technologies, pages 88–99. Springer, 2012.
based features. In Proceedings of the 34th international ACM [40] Y. Zheng, B. Mobasher, and R. Burke. CSLIM: Contextual
SIGIR conference on Research and development in SLIM recommendation algorithms. In Proceedings of the 8th
Information Retrieval, pages 625–634. ACM, 2011. ACM Conference on Recommender Systems, pages 301–304.
[26] A. Odic, M. Tkalcic, J. F. Tasic, and A. Košir. Relevant ACM, 2014.
context in a movie recommender system: UsersâĂŹ opinion [41] Y. Zheng, B. Mobasher, and R. Burke. CARSKit: A
vs. statistical detection. ACM RecSys, 12, 2012. Java-based context-aware recommendation engine. In
[27] C. Ono, Y. Takishima, Y. Motomura, and H. Asoh. Proceedings of the 15th IEEE International Conference on
Context-aware preference model based on a study of Data Mining Workshops. IEEE, 2015.
difference between real and supposed situation data. UMAP, [42] Y. Zheng, B. Mobasher, and R. Burke. Integrating context
9:102–113, 2009. similarity with sparse linear recommendation model. In
[28] U. Panniello, A. Tuzhilin, M. Gorgoglione, C. Palmisano, International Conference on User Modeling, Adaptation,
and A. Pedone. Experimental comparison of pre-vs. and Personalization, pages 370–376. Springer, 2015.
post-filtering approaches in context-aware recommender [43] Y. Zheng, B. Mobasher, and R. Burke. Similarity-based
systems. In Proceedings of the third ACM conference on context-aware recommendation. In International Conference
Recommender systems, pages 265–268. ACM, 2009. on Web Information Systems Engineering, pages 431–447.
[29] R. W. Picard and R. Picard. Affective computing, volume Springer, 2015.
252. MIT press Cambridge, 1997. [44] Y. Zheng, B. Mobasher, and R. Burke. Emotions in
[30] X. Ramirez-Garcia and M. García-Valdez. Post-filtering for a context-aware recommender systems. In Tkalčič,
restaurant context-aware recommender system. In Recent B. Decarolis, M. de Gemmis, A. Košir, and A. Odič, editors,
Advances on Hybrid Approaches for Designing Intelligent Emotions and Personality in Personalized Services, pages
Systems, pages 695–707. Springer, 2014. 311–326. Springer Berlin Heidelberg, 2016.
[31] P. J. Rentfrow and S. D. Gosling. The do re mi’s of everyday [45] Y. Zheng, B. Mobasher, and R. D. Burke. The role of
life: the structure and personality correlates of music emotions in context-aware recommendation. Decisions@
RecSys, 2013:21–28, 2013.