A ective aspects as uncertain variables in decisional processes

Abstract

The importance of emotional aspect in everyday decision is demonstrated by the copious psychological literature about the topic. Emotions are present in every action that we undertake, sure enough when we choose a song to listen we can observe that our decision is inuenced by the a ective state felt in that moment. A consequence of this irrational process is the diversity of choices that we can perform in events that look like equal using observable aspect. The process that a ects the user under a decision is naturally uncertain because there are aspects that involve internal user states like motivational aspects, personal preferences, social expectations and a ective states that are usually automatic and not possible to predict in a logical and rigid approach. In this work we will present a a soft reasoning approach that uses fuzzy rules for predict the emotional coherence of an item for the user during a decisional event.

Keywords

• Emotions
• A ective Computing
• Fuzzy Logic
• Human Decision Making
• Recommender Systems

-

Fuzzy logic is based on the philosophical concept that the reality is not de ned in a precise and rigorous way such as the logical mathematics. This logic proposes a exible interpretation of complexity, vagueness and plurality of the reality. The idea of soft computing was formulated by Lot A. Zadeh[ 11 ], a professor at the University of California, Berkeley, who in 1964 began to realize that the traditional rules of systems analysis were not applicable for treating many of the problems of natural world. The process that a ects the user facing a decision is a natural event that is appropriate for a soft logic as described by Jian [ 16 ]. During this process the a ective aspect is very important because rst of all it has a regulatory e ect that heavy in uences each decision we take. Low intensity emotions have a positive advice role [ 3 ], on the contrary, hight intensity emotions can be a potential source of biases for a clear and logical reasoning [ 2 ]. Moreover, in situations where users have not a complete knowledge of the domain, and the decision can produce risky consequences, negative and intensive emotions like fear and sadness can narrow attention and generate less consciousness decision [ 6 ]. Finally, the user preferences abut elements of domain are strictly dependent by the mood felt in the moment of decision. Emotions are a common natural aspect of our life and they are also vague in their de nitions. The description of what we can consider as emotion is one of the major problems of the eld as described by [ 17 ], that tries to answer at six "perennial problems" that a ect the descriptions of emotion's properties. Recent studies propose the concept of emotion as an holistic outcome from a reasoning process that continuously conceptualizes the internal bodily sensations and sensory input of the immediate situation into emotions (Conceptual Act Theory) [ 18 ]. This view well describes the uncertainty and variability associated with the considered topic. We focus on the following research questions to de ne a fuzzy model to support the emotion formalization and handling: { How is it possible to formalize the emotional state of the user with a fuzzy model that describes it? { How is it possible to reason using soft computing strategy and the fuzzy emotional model of user emotions? 2

Related Works

In literature there are some works that try to formalize emotions by a fuzzy representation. The one proposed by El-Nasr[ 4 ], describes a framework for agents decision making based on strategies of machine learning and the in uence emotions in these processes. El-Nasr[ 4 ] presents a framework "FLAME" that, starting from an external event, computes di erent steps for calculating emotions using a fuzzy logic function de ned as a consequence of expectation and desirability about the event. They are elaborated by the 'behavior selection component' for the arti cial agent that adopts the reasoning system. Fuzzy Models are also used for the human emotions recognition as described by Bakhtiyari[ 1 ]. The author uses this strategy for identifying the 20 PANAS[ 10 ] emotions using data coming from the interaction between user and computer. Similar models are used by Esau[ 5 ] to formalize the emotional state by a dimensional fuzzy hypercube among the dimensions happyness, surprice and sadness in according to the Plutchik[ 7 ] model of emotions. These models will be the fundamentals for this work. 3

Emotions and Fuzzy models

As seen before the decisional process includes di erent aspects that can't be totally calculated and for this reason the idea is to develop systems able to support the user during decisions, suggesting elements that could be appropriate for the situation. Recommender Systems could be used for this task because they are systems able to consider the user preferences through a User model and to suggest elements in accordance with the context and characteristic of items involved. Until now, the cognitive aspects are considered only marginally because the e ort for developing a system able to detect and to use these information, is to high compared with the increase of performance obtained. Instead of these considerations we have seen that a ect in uence is one of the core aspects designed by psychologists as a motivation of a decision. A system able to recognize the user a ective state and to use it (emphatically system) as a parameter of item selection is an important research challenge to resolve. The idea is to de ne a model that is able to use a soft reasoning process for checking if a speci c item is emotionally coherent with the user and context of the decision.

The rst step, for applying this idea, is the identi cation of user mood in the moment of decision. This is important because an item could be interesting for the user in an emotionally state but not in another one. The a ect detection can be done using implicit or explicit strategies. Explicit strategies are based on methods that interact directly with the user asking her the emotion that she feels. Questionnaires can be used to identify user personality traits and user mood. Implicit strategies are based on data obtained monitoring users. One of the most common source used for identi es user's emotions in a speci c moment is video stream, an example in spontaneous context is provided by Tkalcic[ 9 ]. The output of these systems is a percent distribution among a set of nite elements. In accordance with this representation we can assert that the emotional state of the user is de ned as a nite base set of n basic emotions e0, e1, ...en in according to Esau[ 5 ]. Each emotion is described by a level of membership (ei) n in the interval [ 0,1 ] for the fuzzy sets Low, Medium, High with P (ei) = 1. The i=0 emotions adopted are Joy, Sadness, Disgust, Relief, Hope, Fear as described in [ 8 ]. The output of this step is the composition of user's mood as a vector of emotions associated with the correct fuzzy set. The intensity of user's a ective state is another important factor to consider because, as described before, it can in uence the user's perception of the context and preferences of items to choice. For these reasons,it is a good indicator for estimating the emotional coherency of an item. As previous, the value will come from the system of emotion detection that provides a value in an interval [ 0,1 ] easily mapped with a function of membership on the fuzzy sets Low, Medium, High. For deciding if an item is emotionally coherent with the user a ective state it is necessary to identify a main emotion able to represent the item considered. This association is very important because it is useful for moving from the item representation to an emotional domain. In this way the item looses his internal characteristic representation and then it is considered by the system as an emotion. The operation allows to group all the elements that are internally di erent but that generate the same emotion in the user after its consume. The operation could be done using a classi er as Neural Networks or Bayesian's Classi ers learned on domain speci c dataset. For example, in musical domain many works are conducted for the determination of a main emotion associated with a speci c song (Music Emotion Recognition - NER) [12{14]. Sometimes this operation is not easy for the lack of large dataset required for learning optimal rules for the classi cation on an extended set of Emotions. For simplifying this operation we will consider a simple set of category: Positive and Negative. But in more domain a much complex classi cation can be required.

Mood and Intensity of the emotional state are used for the identi cation of the emotional coherence of the considered item. The soft reasoning will use rules as follow:

IF THEN

Joy(U) is A AND Sadness(U) is B AND Disgust(U) is C AND Relief(U) is D AND Hope is E AND Fear is F AND EmotionalIntensity(U) is G AND ItemEmotion(I) is H

EmotionalCoherence(U,I) is L; where U is the User and I is the speci c Item. A defuzzy cation process based on the centroid of the previous results will provide a value between -1 and 1 for emotional coherence that will be used as information about what the user like in that state. This can be used as element of an a ective user pro le used by Recommender Systems able to support the user in decisional tasks. The fuzzy rules necessary to cover all the possible permutation will be learned automatically [ 15 ] by the information that the system will gather from the user interactions in the domain. This is a ground requirement because the number of rules is very high and as previous discussed these reasoning are automatic and di cult to formalize. A problem to face can be the presence of only few interaction data. A solution that is a good agreement between accuracy and data quantity can aggregate users that show a 'similar' personality. This assumption is possible supposing that users, with similar personality,will have an also similar emotional coherence on the same item. 4

Applications in real domains

A systems able to identify and select items, emotionally coherent with the user state, can be used in di erent application scenarios. The most common is in the music and lm domains. When a subject listen a song or watch a movie, emotion are naturally evoked and in this domain it is easy to understand them and use that a ects for suggesting a new element that she could like in that state. If we observed the user with a strategy that assign for each item and mood an emotional coherence value, we can use that elements as seeds for suggesting similar items in new situations. In that scenario, as example, we can adopt the following pipeline. First we will observe the user for some time. We will collect his mood, the intensity of that state and the main emotion of the item selected. The coherence value will be learned observing the user emotional reaction or analyzing his textual feedback. Applying this strategy, after some interactions with the system, we will have all the fuzzy rules that we need for calculating the emotional coherence of a new item in the speci c user emotional state. During the recommendation step, rst we will detect the current user mood and intensity (e.g., joy:high, sadness:low, disgust:low,relief:medium, hope:medium, fear:low; intensity: medium). After that, we will cycle on all the items in catalog and we will check if there are rules that generates an high emotional coherence value. If this happen, all the elements selected will be used by the Recommender System for more ltering operation (e.g. selection considering user explicit genre preferences) and after they will be proposed to the user. A di erent application that concerns video that we can imagine is for advertising purposes. If we can detect the user mood rst of the watching and also during all the duration of a video we can identify his positive and negative emotions during it. In this way we can understand the correlation among the user mood, the intensity of that state and the items. As rst we can generate the correspondent fuzzy rules and use them for suggest emotionally coherent elements. 5

Evaluation planning

The evaluation of systems that work with user personalization and more in the speci c with user emotions can be really di cult to do, and an "in vitro" evaluation can be done only if datasets with enough information are available, but until now,as much of my knowledge, they are not public. For this reason we are going to planning an "in vivo" evaluation. The idea is to use our music recommendation system for providing music suggestions to the user. In the rst step we will detect the user mood and intensity, we will ask to provide us ve songs that she likes in that moment and after we will provide her some new songs coherent with the fuzzy rules learned. For each song we will ask her if she like it; if she know it; if she consider it coherent with her current mood. The data gathered will be used for evaluating the accuracy of the model and the ability to generate serendipity. 6

Remarks and Ongoing Work

Emotions are an important aspect that in uences people's behaviours and life events. When a person faces a decision task, like the choose of a lm, a song or a nancial investment, she performs a cognitive process to choose the solution that will generate positive consequence. In this process, desirability and expectation, detected from the user emotions, in uence the nal decision. The proposed approach to the problem promotes the area of computer science that uses the soft reasoning as a strategy for formalizing all the problems that involve the natural vagueness. As described, the emotional in uence in the human decisional process is well representable by this kind of assumptions. The fuzzy rules proposed resume partially the role of mood and emotions in the decision event obtaining an emotional coherence value usable in system that can support the user during this task, like Recommender Systems. The de nition of a complete computational model of emotions for Emotion Aware recommender Systems is the purpose of the author's doctoral studies.

Acknowledgment

This research has received funding from the European Unions Horizon 2020 research and innovation programme under the Marie Skodowska-Curie grant agreement No 691071.

References

1. Kaveh Bakhtiyari and Ha zah Husain: "Fuzzy Model in Human Emotions Recognition." arXiv preprint arXiv:1407.1474 , 2014 .

2. Roy F. Baumeister , Todd F. Heatherton , and Dianne M. Tice : "Losing control: How and why people fail at self-regulation . " Academic Press, 1994 .

3. Antonio R. Damasio: "Descartes Error: Emotion, reason and the human brain." Penguin Group (USA), Inc . 336 pages, 1994 .

4. Magy Seif El-Nasr, John Yen, and Thomas R. Ioerger :"Flamefuzzy logic adaptive model of emotions." Autonomous Agents and Multi-agent systems 3.3 , Pages: 219 - 257 , 2000 .

5. Natascha Esau , Lisa Kleinjohann, and Bernd Kleinjohann: "An Adaptable Fuzzy Emotion Model for Emotion Recognition." EUSFLAT Conf . , 2005 .

6. Joseph P. Forgas : "Mood and judgment: the a ect infusion model (AIM)." Psychological bulletin 117.1 Pages: 39 , 1995 .

7. Robert Plutchik : "The emotions ." University Press of America, 1991 .

8. Donald D. Price , James E. Barrell , and James J. Barrell : "A quantitativeexperiential analysis of human emotions." Motivation and Emotion 9.1 , Pages: 19 - 38 , 1985 .

9. Marko Tkalcic , A. Odic , A. Kosir : "The impact of weak ground truth and facial expressiveness on a ect detection accuracy from time-continuous videos of facial expressions." Information Sciences , 249 . Pages: 1323 , 2013 .

10. David Watson, Lee A . Clark , and Auke Tellegen: "Development and validation of brief measures of positive and negative a ect: the PANAS scales . " Journal of personality and social psychology 54.6 , 1988 .

11. Lot A . Zadeh: "Fuzzy sets." Information and control 8 .3, Pages: 338 - 353 , 1965 .

12. Babu Kaji Baniya and Choong Seon Hong: "Music Mood Classi cation using Reduced Audio Features." Pages : 915 - 917 , 1995 .

13. Jaesung Lee et al. : "Investigating Relation of Music Data: Emotion and Audio Signals . " Computer Science and its Applications . Springer Berlin Heidelberg, Pages: 251 - 256 , 2015 .

14. Youngmoo E. Kim et al. : "Music emotion recognition: A state of the art review . " Proceedings of ISMIR , 2010 .

15. Robert J. Hammell and Thomas Sudkamp: "Learning fuzzy rules from data." The Application of INformation Technologies (Computer Science ) to Mission Systems, 1998 .

16. Lirong Jian , Sifeng Liu, and Yi Lin: "Hybrid rough sets and applications in uncertain decision-making." CRC Press, 2010 .

17. Magda B. Arnold : "Perennial problems in the eld of emotion." Feelings and emotions: The Loyola symposium . New York: Academic Press, 1970 .

18. Lisa Feldman Barrett: "The conceptual act theory: A prcis." Emotion Review 6.4 , Pages: 292 - 297 , 2014 .