The in uence of user's emotions in recommender systems for decision making processes

Abstract

The decision making process is a very hard task to face, because a lot of external elements could in uence the nal decision taken. The paper will present the in uences of emotions and personality in this task and will propose a recommender system able to take them in accounts during the recommendation process.

Keywords

• Emotions
• Recommender Systems
• Human Decion Making

Introduction

People have to take decisions, about tasks they have to carry out, everyday. Sometimes, decisions are easy to be taken because the context is usual or the generated consequences are not crucial. Other times, decisions are very di cult to be managed and emotions like fear, sadness or surprise will in uence negatively the logical reasoning. Emotions play a cardinal role in the decision making task [ 1 ] and their in uences should be taken into account during the recommendation process to propose solutions that could improve the positive emotional state of the user.

Traditional recommender systems do not include emotions in the computational process and only recently some works have explained how is possible recognizing personality and emotions from di erent sources and apply them. Gosling shows how to extract the personality pro le from Facebook [ 2 ], and Golbeck uses Twitter for the same task [ 3 ]. The authors describe how to adapt the recommender output to the user personality pro le but they do not consider emotions. Tkalcic[ 5 ], suggests using emotions in di erent stages of recommendation process but how changing the process in each stage is an open issue. An user is very in uenced by incidental emotions [ 4 ] and by expected consequences that the decision will produce on the future emotional state when he faces a new and relevant decision task. A recommender system should recognize this emotion, and should adapt, starting from user preferences and user personality traits, the recommendation output in according to the user emotions to help her taking a clear and a pondered decision.

This early stage work, supervised by Marco de Gemmis 1 , proposes a strategy to consider emotions in the recommendation process. The main research topic that we will investigate are: 1. Contextualization of emotional and personality traits that in uence the decision task; 2. De nition of a computational emotions model useful for recommender systems; 3. Identi cation of emotional source to use during the recommendation process; 4. Study of innovative recommender system models based on emotional pro les. Later in the document we describe the preliminary idea and how the future results will contribute to improve the current recommender systems state of the art. 2

Related Works

Emotions have been studied ample in the psychological and cognition areas. The rst fundamental study was conducted by Darwin[ 12 ], who supposes that emotions are universal in according to their origin of the species theory. Ekman [ 8 ] in uenced by Darwin, identi es six universal emotions (happiness, sadness, surprise, fear, disgust and anger) and their respective facial micro expression. This classi cation is used largely in di erent domains, and their experimental results are fundamental for the emotion recognition task of this work. Important studies in psychology about the in uence of emotions in a decision making task are available. Norbert Schwarz [ 10 ], provides a selected discussion about emotion, cognition and decision. The author recaps fundamental concepts to consider while people face with the decision making task, that is post decision a ects, anticipated a ects and memories of past a ects. This concepts are largely used in this work.

The interest of the research about the use of emotions in computer science is more recent, particularly in recommender systems area. Zheng and Burke [ 11 ] demonstrate how emotions are relevant in the context-aware recommender systems, analogous results were obtained from Tkalcic[ 6 ] that demonstrates how a ective labelling increases the performances of content base recommender systems. This results show that taking into account emotions in recommender systems, generates bene ts. In according to this we will work on this area to contribute to generate relevant results for the topic. 3

Approach

The main objective of the work is to de ne a framework that includes the emotional aspects into recommender systems, and particularly in which that support 1 Marco de Gemmis, [e-mail], Dept. of Computer Science, University of Bari Aldo Moro, Italy Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.

the decision making process. The work is in an early stage, and more studies and in-depth analysis will be conducted. To solve the proposed problem, we decided to face it identifying sub-tasks that focus on di erent aspects.

First, we will focus on de ning an emotional model for identifying, formalizing and classifying user emotions. Consequently we will de ne an user emotional pro

le that includes informations to be used in recommender systems. Finally, we will identify behaviours of the recommender systems in according to the de ned emotion pro le. 4

Identi cation of Emotions

Emotions are fundamental elements of the cognitive research area and a great deal of models has been prosed. We adopt the Ekman model [ 8 ] of emotion because the low number of primary emotions de ned in it supports the combination of di erent strategies of emotions identi cation.

These strategies can be implicitely or explicitely. The explicit strategy is invasive because the system will ask users about emotions that they feel. The elicitation of emotions is not an easy task. People, often, do not really know what they really feel. Questionnaires for deducing emotions and personality traits could be adopted [ 14 ]. The implicit strategies are not invasive because all the collected data do not in uence the natural behaviours of the user, but they are a ected by a lot of imprecisions in spontaneous context [ 7 ].

We will detect a ects in three di erent stages of user computer interaction and in each one we will use a speci c type of emotion identi cation strategy in according to Tkalcic's [ 5 ] work. 1. Early stage: incidental a ects will be detected analysing latest daily posts on

Facebook and Twitter. Techniques of Natural Language Processing (NLP) will be used to make a quickly emotional analysis of the latest user's shares. 2. Consumption stage: expected consequences of the decision will generate evident emotions. Will be used techniques of on-line video analysis combined with an explicit question about emotion can generate an accurate emotionally user pro le. We will obtain a single value from a questionnaire and a distribution of percentage value among the six basic emotions from the video analysis. In this stage user emotion pro le will be extended with information obtained by a proposed 44-items Big Five Inventory questionnaire[ 14 ].

This questionnaire will be proposed only the rst time that the system will interact with the user. 3. Exit Stage: after the decision, the consequential emotion will be generated.

This step is used to get a feedback from the user about the decision taken.

If consequences of decision are not immediate, an analysis of social network posts, using the same techniques adopted in early stage emotion identi cation, will be done when consequences will occur.

Using Social Networks posts it is possible to identify user personality traits [ 2, 3 ] and user emotional state [ 13, 20 ] adopting strategies of Natural Language Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.

Processing. Several strategies are adopted in machine learning literatures for this task and one of the most useful framework adopted is SNoW, a general purpose multi-class classi er [ 22 ]. Strategies based on emotion lexicon are also popular. A general strategy that use this resource, rst, identi es key terms of the sentence and, later, checks the emotions associated to each world in a emotion based lexicon [ 23 ]. The speci c algorithms of NLP, the questionnaires to propose to the user and framework of video-audio analysis to adopt are object of evaluations

The speci c algorithms of NLP, the questionnaires to provide and framework of video analysis to adopt are not currently decided. 5

A ective pro le

The user emotions gathered during the decision making are formalized and stored in an e ective user pro le. The pro le will be used by the recommender system to adapt its computational process and to generate recommendations in according to emotions. It is composed by di erent elements: user personality traits (PT), historical decision cases (HC), contexts and user expertise (CE).

AP = P T

HC

CE (1) Personality Traits. Personality traits are formalized as a distribution of percent values among the dimensions: Openness to experience, Conscientiousness, Extroversion, Agreeableness, Neuroticism in according to the Big Five model [ 15 ].

Historical decision case. An historical decision case describes accurately the decision making task and emotions felt by the users. A case contains early stage emotions, consuming stage emotions, exit stage emotions, and a description of the task.

Early stage emotions in according to the description before provided, are formalized as a distribution of percent values among the six emotions of Ekman model[ 8 ] gathered from social networks posts. The same formalization is used for the consuming stage and exit stage emotions. In the consumption stage emotions is gathered an additional value from user asking the emotion she felt. The description of the task is de ned by: context of decision, problem, elements among which choosing, decision taken, explicating feedbacks in a scale from 1 to 10 to describe the utility of suggestions (1 not useful, 10 extremely useful). The historical case could be enriched with more features, for example a description of interaction between user and system, but we decide to simplify the situation for realizing a preliminary working framework.

Context and expertise. The context is characterized by explicit features that describe user preferences in this domain. The expertise of user in the speci c domain is the number of decisions taken in this context, starting from an initial Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.

value obtained from a user ability questionnaire. The available contexts of applications will be de ned a priori in a list of chooses because we do not focus on the context detection strategies.

Memory of the past a ect Each time that the user will take a decision, interacting with the system, her a ective pro le increases and new emotionally historical cases will be added. Some cases are more important than others because they generate negative emotions that in uence future user decisions in the same domain more than a positive emotion. This e ect is called memory of the past a ect and it is well described by LeDoux [ 16 ].This e ect will be considered in the framework to provide, in this particular case, only solutions that generate positive emotions to increase the initial user mistrusts. 6

Emotions and Recommender Systems

Recommender Systems (RSs) are largely used in a lot of di erent domains, from the classical e-commerce system to the more di cult and risky nancial advisory domain. An important application of them is in the decision making process.

When people have to take a decision, they have to analyse the situation and to take a logical choose among di erent options. A RS can support this task, proposing solutions in according to user necessity, for example, obtaining them from similar users decisions. In contexts where it is important making logical decisions, proposed solutions have to be full demonstrable, because a motivation of the process used to select each proposal have to be provided. A recommender system have to adopt di erent behaviours in according to the context of decision to provide suitable recommendations. We identify three di erent kinds of contexts varying the risk value among hight, median and low in according to the risky consequence that the decision could generate. A decision is risky if it heavily in uence user common behaviours and user has low experience in the domain. For example decisions about money or health can be risky decisions. 1. High risk: they are decisions hard to be taken. In this context it is important providing correct and demonstrable solutions. The core aspect to be taken into account is the correctness. An application that occurs in this context is a recommender systems for health wellness. 2. Medium risk: they are decisions that in uence behaviours momently without the possibility to revert them. An application that occurs in this context is a recommender systems for fashion shopping. 3. Low risk: they are decisions that the user makes commonly and that are easy to revert. Working on this context allows to suggest new and uncommon elements diversifying users decisions. An application that occurs in this context is a music recommender system. Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.

Using this de nitions of context, it is possible to de ne a general framework for recommender systems able to include emotions in the recommendation process (Fig. 1). The Emotion Aware Recommender System, starts the recommendation process from personality traits. Users with similar personality and explicit preferences in same contexts, are detected using common similarity measures, for example a cosine similarity on vector obtained combining pro le features and preference features. The system uses this set of users to identify decisions taken in the past for the considered problem. This set of solutions have to be taken in same emotional state of the considered user and, it has positive exit stage emotion or explicit positive feedback of utility. This solutions will be ltered or ranked in according to speci c recommendation process in uenced by the application context. The output generated is a list of possible solutions to the problem in uenced by emotionally attribute. During the three stages of the process, emotions are identi ed in according to the process described previously.

All the informations about the decision taken will be stored inside an historical emotion case that will update the user a ective pro le. The described recommendation strategy is base on the Case-based reasoning [ 21 ] one of the most commonly adopted machine learning method, that exploits a knowledge-based representation of the context. The formalization of the computational process of recommender system that face emotions into di erent contexts, is today being studied, but some preliminary ideas will be provided.

If not enough data will be available for this computation, an inference based only from personality traits in the speci c domain will be done, but this task is now an open issue of the work. 6.1

Recommendation process in hight risk context

The recommendations about decisions that will generate risky consequence have to be accurate and demonstrable. For this kind of tasks, a rule base system could be a good solutions. The knowledge about domain could be fully encoded and combined with speci c rules that consider emotions. A demonstration of Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.

the proposed elements could be always provided. A set of possible general rules about early stage emotions, is provided. 1. Happiness and Neutral: the user is relaxed and used to face the problem [ 18 ]. The system has to maximize user pro t. Users historical solutions can be included, if possible. It is possible to diversi er solutions between low and hight probability to produce negative consequences. 2. Sadness and Anger: the user is not fully concentrated [ 17 ] on the task. The system has to prefer a good agreement between user pro t and low probability to generate negative consequence. If successful historical user cases are presents, they could be adopted re ning the proposed solution. 3. Surprise, fear and disgust: the user maybe does not have enough expertise to take a safe and accurate decision. The system has to fully describe each solution provided and to propose an element with low probability to generate negative consequence.

If a large number of negative a ect memories is present in the user history, the system has to increase the con dence with the user providing only solutions that certainly produce positive emotions. A formalization of presented rules will be realized later in research. 6.2

Recommendation process in medium risk context

Medium risk decisions provide more degree of freedom than risky decisions, because it is not mandatory to provide a demonstration of the solutions. It is possible to mediate recommendations providing items that strongly meets user preferences and also items that are in according to user emotions. A description of pros and cons of each proposed solution could be useful to support the user decision. 6.3

Recommendation process in low risk context

Emotions play a relevant role in low risk contexts [ 9 ] because decisions are often taken without a long and logical reasoning. Incidental emotions in uence intensively the decision making task and people take decisions in according to them.

Each person has di erent behaviours in this situation, for example some people like listening sad jazz music, others like happy jazz music and other ones generic rock music when they are sad. In this context recommendations are based on the user historical cases and the similar users decisions in the same emotionally state. Particularly, if the user is in an happiness state, an intensive diversi cation of results could be done, trying to generate serendipity. If historical cases are not available, it is possible to consider the possibility of import user emotionally preference from correlated similar domains. For example the study of Cantador[ 19 ] demonstrates how it is possible to extend books preferences in music or lm domain. Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.

Conclusions and future work

Emotions are important elements of people's life. In each decision making task, emotions in uence the nal option choose. In contexts that generate riskly consequence they need to be mitigated, in others, for example music domains, they can be ampli ed and used to generate useful suggests. Systems that support the decision making task, currently do not consider properly emotions, for this reason, we have proposed a framework able to include emotions inside recommender system process. It could be used to produce recommendations in according to the user preferences, personality and emotional state. The ideas proposed in this paper are consequences of a preliminary work on the topic that need more studies and empirical experiments to formalize them. Open issues are presents and a lot of elements need clari cations but this overview could provide useful global schema for a future complete work. Proc. of CHItaly 2015 Doctoral Consortium, Rome (Italy), September 28th 2015 (published at http://ceur-ws.org). Copyright © 2015 for the individual papers by the papers' authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors.

References

1. Antonio R. Damasio: "Descartes Error: Emotion, reason and the human brain." Penguin Group (USA), Inc . 336 pages, 1994

2. Samuel D. Gosling , Sam Gaddis, Simine Vazire: "Personality Impressions Based on Facebook Pro les." ICWSM 7 , Pages : 1 - 4 , 2007

3. Jennifer Golbeck , Cristina Robles, Michon Edmondson, Karen Turner: "Predicting Personality from Twitter." SocialCom/ PASSAT Pages: 149 - 156 , 2011

4. Gerald L. Clore : "Cognitive Phenomenology: Feelings and the Construction of Judgment." In L. Martin. Tesser (eds.) , The Construction of Social Judgments . Lawrence Erlbaum. Pages: 10 - 133 , 1992

5. Marko Tkalcic , Andrej Kosir and Jurij Tasic: "A ective recommender systems: the role of emotions in recommender systems." RecSys11 Workshop on Human Decision Making in Recommender Systems In conjunction with the 5th ACM Conference on Recommender Systems , Chicago, IL, USA, October 23 - 27 , 2011

6. Marko Tkalcic , Ante Odic, bAndrej Kosir, Jurij Tasic: "A ective Labeling in a Content-Based Recommender System for Images." IEEE transactions on multimedia , vol. 15 , no. 2 , February 2013

7. Marko Tkalcic , Ante Odic, Andrej Kosir: "The impact of weak ground truth and facial expressiveness on a ect detection accuracy from time-continuous videos of facial expressions . " Inf. Sci. 249 , Pages: 13 - 23 , 2013

8. Paul Ekman : " Facial expression and emotion." American Psychologist , 1993

9. Norbert Schwarz : "Feelings as information: informational and motivational functions of a ective states . " Guilford Press, 1990 .

10. Norbert Schwarz: "Emotion, cognition, and decision making." Cognition & Emotion Volume 14, Issue 4 , pages 433 - 440 , 2000

11. Yong Zheng , Bamshad Mobasher, and Robin D. Burke . "The Role of Emotions in Context-aware Recommendation." Decisions@ RecSys, 2013

12. Charles R. Darwin : "The expression of the emotions in man and animals . " London: John Murray. 1st edition , 1872

13. Carlo Strapparava , Rada Mihalcea: "Learning to identify emotions in text . " SAC , Pages: 1556 - 1560 , 2008

14. Oliver P. John , Sanjay Srivastava: "The Big Five trait taxonomy: History, measurement, and theoretical perspectives." Handbook of personality: Theory and research 2 . Pages: 102 - 138 , 1999 .

15. Lewis R. Goldberg : "An alternative description of personality: the big- ve factor structure . " Journal of personality and social psychology 59.6 , 1990

16. Joseph E. LeDoux : "Emotional memory systems in the brain . " Behavioural brain research 58.1 , Pages: 69 - 79 , 1993

17. Michael R. Basso et al. : "Mood and global-local visual processing . " Journal of the International Neuropsychological Society , 2 ( 03 ), Pages: 249 - 255 , 1996

18. Galen V. Bodenhausen , Geo rey P. Kramer , and Karin Susser: "Happiness and stereotypic thinking in social judgment . " Journal of personality and social psychology 66.4 , Page: 621 , 1994

19. Ivan Cantador , Ignacio Fernandez-Tobias, and Alejandro Bellogin: "Relating personality types with user preferences in multiple entertainment domains . " CEUR Workshop Proceedings. Shlomo Berkovsky , 2013

20. Alexander Pak and Patrick Paroubek: "Twitter as a Corpus for Sentiment Analysis and Opinion Mining . " LREC . Vol. 10 . 2010

21. Agnar Aamodt and E. Plaza : "Case-based reasoning: foundational issues, methodological variations, and system approaches . " AI Communications , 7 ( 1 ): 3959 , 1994

22. Cecilia Ovesdotter Alm, Dan Roth, and Richard Sproat: "Emotions from text: machine learning for text-based emotion prediction . " Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing. Association for Computational Linguistics , 2005

23. Alexandra Balahur , Jesus M. Hermida , and Andres Montoyo: "Building and exploiting emotinet, a knowledge base for emotion detection based on the appraisal theory model." A ective Computing , IEEE Transactions on 3.1 . Pages: 88 - 101 , 2012