Application of Lovheim Model for Emotion Detection in English Tweets

Abstract

-Emotions are central for a wide range of everyday human experiences and understanding emotions is a key problem both in the business world and in the fields of physiology and neuroscience. The most well-known theory of emotions proposes a categorical system of emotion classification, where emotions are classified as discrete entities, while psychologists say that in general man will hardly express a single basic emotion. According to this observation, alternative models have been developed, which define multiple dimensions corresponding to various parameters and specify emotions along those dimensions. Recently, one of the most used models in affective computing is the Lovheim's cube of emotions, i.e., a theoretical model that focuses on the interactions of monoamine neurotransmitters and emotions. This work presents a comparison between a single automatic classifier able to recognize the basic emotions proposed in the Lovheim's cube and a set of independent binary classifiers, each one able to recognize a single dimension of the Lovehim's cube. The application of this model has determined a notable improvement of results: in fact, in the best case there is an increment of the accuracy of 11,8%. The set of classifiers has been modeled and deployed on the distributed ActoDeS application architecture. This implementation improves the computational performance and it eases the system reconfiguration and its ability to recognize particular situations, consisting of particular combinations of basic emotions. Index Terms-Machine learning, sentiment analysis, emotion detection, natural language processing, software actors.

I. INTRODUCTION

Sentiment Analysis (SA) applies Machine Learning (ML) techniques to textual content, for extracting the feelings and other information, useful for understanding a person’s opinion about a given entity (product, person, topic, etc.). Emotions are central for a wide range of everyday human experiences and understanding emotions is a key problem in the business world, especially in an era where online communities will define future products and services [ 1 ], [ 2 ].

Mainly in market analysis, being able to discriminate between positive and negative comments is often sufficient. But, for other applications that refer to Affective Sciences (AS), a more detailed analysis of emotions is more appropriate. As refer to the scientific study of emotion (or affect) and cover a wide range of interdisciplinary fields in which emotions play a fundamental role, such as medicine [ 3 ]. Emotions represent and identify the human being and therefore they influence a person’s behavior and decisions and also the relationships with others [ 4 ]. In evolutionary terms, their main function is to make the individual’s reaction more effective. in situations where an immediate response is needed for survival. Such a reaction does not use cognitive processes and conscious processing. An interesting theme in physiology and neuroscience is the question of how emotions interact with and influence other domains of cognition, in particular, attention, memory, and reasoning [ 5 ]. In fact, the question of how emotions are represented in nervous system activity is still an unresolved problem in affective neuroscience [ 6 ].

The most well-known theory of emotions proposes a categorical system of emotion classification, where emotions are classified as discrete entities, independent of each other and easily distinguishable [ 7 ]. Thus the main taxonomies of emotions divide them into positive and negative and more into details in a few basic emotions: surprise, interest, joy, rage, fear, disgust, shame, and anguish [ 8 ]. In reality, psychologists and social science experts say that in general man will hardly express a single basic emotion [ 9 ]. Every facial expression, every text, every gesture is a composition of multiple emotions. Such expressions change together with the evolution of speech. Similarly to the different shades of color, emotions are not clearly distinguishable and it is difficult to sharply discern one emotion from the others. In argumentations about the human personality, even the objective and neutral information, typically used to classify simple information about an object or a theme, cannot be expressed in their pure form.

Recently, one of the most used models in affective computing is the Lovheim’s cube of emotions ( [ 10 ], [ 11 ]), that is a theoretical model that focuses on the interactions of monoamine neurotransmitters and emotions. Although the validity and reliability of this model are still to be determined, some researches demonstrate that a neurocognitive approach is important to determine emotional reactions to, for example, visual stimuli [ 12 ].

This work presents a comparison between a single automatic classifier able to recognize the basic emotions proposed in the Lovheim’s cube and a set of independent binary classifiers, each one able to recognize a single dimension of the Lovehim’s cube. A set of classifiers has been modeled on the actor-based architecture proposed in [ 13 ] for data analysis. This implementation improves the computational performance and it eases the system reconfiguration, to recognize particular situations, that is particular combinations of percentages of basic emotions. In fact in the Lovehim cube, the basic emotions are classified on a three-dimensional model that allows highlighting their reciprocal relations, which difficult to distinguish in a hierarchical model, in which all emotions are first of all divided into negative, neutral and positive (see for example [ 14 ].

A curiosity, and a difficulty in the implementation of this model for the recognition of emotions on Twitter, was the lack of available data for the “distress / anguish” emotion. That class is commonly found in the datasets for facial emotion recognition but not in textual datasets, probably due to the approach of the human beings towards the written text [ 15 ].

The paper is structured as follows. Section 2 gives a brief overview of the background in emotion detection. Section 3 describes the implemented framework and Section 4 describes the experimental results. Section 5 reports the concluding remarks.

II. RELATED WORK

The techniques of Sentiment Analysis aim at the study and analysis of textual information, with the purpose of detecting evaluations, opinions and emotions related to a specific entity (product, person, topic, etc.). This type of analysis has important applications in the political, economic and social fields, such as Web Reputation and Social Media Analytics. Social media and the rise of social networking platforms are one of the most important social phenomena in the last years [ 16 ] and the studies of SA on platforms like Twitter are very common in the scientific literature ( [ 17 ]–[ 19 ]. Datasets based on tweets are used in different tasks of Semantic Evaluation (SemEval), such as in [ 20 ] and [ 21 ].

SA, with thousands of articles written about its methods and applications (see for example [ 22 ]–[ 24 ], is a well-established field in Natural Language Processing. Emotion Detection can be viewed as a natural evolution of Sentiment Analysis and its more fine-grained model [ 25 ]. This paper summarizes the emotion models that are mostly used in emotion-based research.

Emotion extraction from different types of social network components is a research topic which is being investigated for a long time now. An updated more comprehensive survey of Emotion Detection from a text can be found in [ 26 ].

Some works in the literature for emotion detection have an approach based on lexicon or keyword and in this case they refer to an annotated dictionary or a knowledge base; but many recent works adopt a machine learning approach [ 27 ], in this work we use a framework that is based on supervised machine learning algorithms.

Most of the works in the literature refer to the theory of the six primary emotions: in general, a text is recognized as belonging to one of these emotions by an automatic classifier.

In a previous work of ours [ 14 ] we used a hierarchical approach so that first we looked for the polarity of a text and then the primary emotion associated with it. This hierarchy of classifiers has made the system more flexible and slightly improved the performance respect to a flat classifier trained on the same dataset based on six emotions. In the present work, we have tried to exploit a more recent and more refined theory of emotions that is able to perform a multidimensional analysis of them. Some work of this kind is present in [ 28 ], even if the authors used a two-dimensional model, while our reference model has three dimensions: the different emotions are inside a cube whose vertices are the eight "pure" emotions.

Also, in this case, the system is more flexible than a flat classifier able to classify the eight primary emotions as it is possible to define a set of points in our multidimensional model that represent combinations of “pure” emotions to associate "real" human moods. Furthermore, in this case, the comparison of the flat classifier trained on the same dataset gave significantly better results. It is not possible to make a direct comparison between the two systems as they refer to different datasets and to different data collection approaches.

The models of Parrot, Plutchik, and Lovheim are shown below. Parrot’s model belongs to the set of categorical emotion models which define a list of discrete categories of emotions.

Parrott proposed an emotion classification model structured on three levels, each representing a list of emotions [ 29 ]. The model collects more than 100 emotions and is shown in Figure 1.

The other two models belong to the class of dimensional emotion models, which define a few dimensions with some parameters and specify emotions according to those dimensions.

Plutchik argued that there are only eight basic emotions (joy, trust, fear, surprise, sadness, anticipation, anger, and disgust), but these emotions can be combined. He suggested the wheel model (2D) in 1980 to describe how emotions are related [ 30 ]. The model proposed by Plutchik is summarized in Figure 2. It is possible to observe that this model describes a wide spectrum of emotions, each of them representing a different combination of primary emotions.

Finally, Lovheim proposed a three-dimensional model based on the relationship between neurotransmitters and emotions [ 10 ]. In the model, the three neurotransmitters serotonin, dopamine and noradrenaline form the axes of a coordinate system, while the eight basic emotions (shame, anxiety, fear, anger, disgust, surprise, joy, and interest) are placed in the eight vertices. Each vertex of the cube corresponds to one of the eight possible combinations of the three neurotransmitters, as shown in Figure 3. The relationship between emotions and neurotransmitters is shown in Figure 4.

Neurotransmitters represent a certain type of hormones that affect the amino acids located in the brain and transmit information from one neuron to another. To gain a better understanding of how neurotransmitters have an effect on our emotions, a brief explanation of their functioning is needed.

An electrical signal (or nerve impulse) travels along the neural pathways until it reaches their end. Once the end of the path is reached, the electrical signal is transformed into a chemical signal (neurotransmitter) inside the synapses (space between neurons). This signal, crossing the synapses, will be transformed back into an electrical signal. The action potential, a phenomenon that occurs when the energy of a cell rapidly grows/decreases , starts the release of these neurotransmitters from the presynaptic terminal nerve through a process of exocytosis: a cellular process by which a cell expels molecules.

Neurotransmitters are packed into vesicles in the presynaptic neuron. Once released, they enter the synapse by attaching to receptors in the post-synaptic neuron. In biology, the neurotransmitters considered by Lovehim in relation to emotions have the following functions: 1) Serotonin, also known as the happiness hormone, deals with numerous functions such as regulation of circadian rhythms (sleep), appetite control, blood pressure control, control of sexual behavior. It has a positive effect on memory, has inhibitory effects on the perception of pain, and intervenes in social relations. Low levels of serotonin lead to a decline in mood, depression, states of anxiety, and aggression. High levels of serotonin instead determine a state of well-being, serenity, tranquillity, and happiness. 2) Dopamine deals with the following functions: movement, memory, sleep, mood, learning, attention, and reward. An excess or deficiency of dopamine is the cause of numerous diseases such as Parkinson’s, drug addiction and schizophrenia. 3) Noradrenaline is released by the brain in response to strong physical or psychological stress. It accelerates the heart rate, increases the release of glucose from energy reserves, and increases blood flow. Noradrenaline also intervenes in the preparation of the body in the so-called attack or flight reaction. Noradrenaline is the neurotransmitter of excitement: low levels are related to depression, poor memory, lower than average alert levels. Too high levels are linked to increased anxiety and fear.

One of the main targets of this study is to compare results obtained by a FLAT classifier, trained using the training set directly, with results obtained by a classifier trained using a

Lovheim architecture and a corresponding training set. The III. CLASSIFICATION SYSTEM file modeled according to Lovheim’s theory is split into 3 binary training sets, according to high/low levels of the three

This chapter describes the structure of the realized classi- characteristic neurotransmitters: serotonine, dopamine, norafication system, including the preparation of the dataset, the drenaline. Then, three binary classifiers are trained and their learning techniques, and the actor-based architecture. results are combined and compared with those obtained from the simple FLAT classifier. Moreover, since in all publicly A. Actor-based System available dataset there are only 7 out of the 8 emotions of Lovheim’s Cube, an assumption has been adopted to avoid

For realizing this kind of multilevel classification system, the problem. we have used ActoDES, which is a software framework which The dataset contains a collection of tweets (22900), which adopts the actor model for simplifying the development of address different topics. Each tweet is labeled with one of the complex distributed systems [ 13 ], [ 31 ]. In fact, it eases the following emotions: sadness, worry, hate, boredom, surprise, creation of complex intelligent systems, supporting the exe- happiness, fun, relief, love, enthusiasm. cution of multiple autonomous classifiers which communicate Since the annotated classes do not correspond to Lovheim’s by means of asynchronous messages. This way, a composite basic emotions, another step is needed before training files classifier can be deployed as a set of loosely coupled cooper- can be created: mapping the dataset emotions onto Lovheim’s ating actors. Each simple classifier and each processing step cube (3). So the emotion classifications proposed by Parrott can instantiated as an actor, allowing the whole architecture and Plutchik have been used. The list of mapped emotions is to be defined at a high level of abstraction, where single shown in table I. actors can be replaced and reconfigured to evaluate alternative approaches. According to the received messages, an actor IV. TRAINING SETS can update its state and change its behaviors, terminate its own execution, send messages to other actors, create new After the mapping has been completed, 4 training files have actors, etc. Particular interaction patterns allow communities of been created: actors to self organize in dynamic scenarios, involving online • 1 FLAT Training set: this file has 22900 tweets, where learning. A subscription service is available in ActoDES, to each one is labelled with one of the following emotions: facilitate the development of collaborative applications with Shame, Fear, Anger, Disgust, Surprise, Joy, Excitement. actors, as shown in [ 32 ]. It is very useful for the integration of • 3 BINARY Training sets: these files have 22900 tweets, multiple actors participating in a structured classification task. where each one is labeled with 1/0 which indicates the Other services, developed for this project, provide additional high/low level of the neurotransmitter for the correspondfunctionalities to actors, for the continuous analysis of various ing Lovheim emotion. social streams. E.g., let us consider the following tweet: "wants to hang

Leveraging ActoDES and the additional mentioned services, out with friends soon!" we have built a software system which can be used to track This tweet is labeled with the interest/excitement emotion and study a news feed from social media, with an architecture in the FLAT training set. According to Lovheim’s theory, that can be extended to different cases and also to more this emotion has the following coordinates in the cube complex problems. In particular, it can be configured for online (1,1,1), which correspond to high levels of serotonine, operation, handling streams of messages and continuously dopamine and noradrenaline. So, this tweet has been learning from a growing dataset. labeled in each binary training sets with a 1.

For each training file, a quite classical set of preprocessing filters is applied. The configuration is obtained after an iterative optimization process. In particular, sentences are vectorized according to the bag of words approach, after applying the Iterated Lovins stemmer, stopwords removal, and tokenization for unigrams and bigrams. Finally, the most relevant and useful features are selected for the resulting data, according to the InfoGain algorithm, with a zero threshold.

V. EXPERIMENTAL RESULTS

This chapter presents the results retrieved applying the different compared classification systems, supervised approach using the 10-folds-cross validation methodology on the different training sets.

A. FLAT training set

The results for the FLAT training set are shown in table II, while its confusion matrix is shown in table V.

B. Binary training sets

The results obtained from each binary training file would be irrelevant if taken individually: it is necessary to consider the combination of the predictions on these 3 datasets. The combined results are shown in table III.

The corresponding confusion matrix is shown in table VI.

C. Assumption

Since a dataset which contains at least an emotion that could be labeled as distress/anguish has not been found, an escamotage has been adopted to remove predictions that could have been classified as this class, because it wasn’t present in the training file.

This problem arises because the results are obtained from the 8 combinations of serotonine, dopamine and noradrenaline levels: it could happen that the following combination (0,0,1) is predicted, which corresponds to the distress/anguish emotion in Lovheim’s cube.

The following assumption is adopted: considering the predictions and the confidence levels associated with each tweet, classify the text with the closest emotion. Table IV shows the neurotransmitters results, after applying the assumption. In table VII, the related confusion matrix is shown.

VI. CONCLUSIONS

Recently, one of the most used models in affective computing is the Lovheim’s cube of emotions: a theoretical model that focuses on the interactions of monoamine neurotransmitters and emotions. This research work has compared a single automatic classifier, able to recognize the basic emotions proposed in the Lovheim’s cube, with a set of independent binary classifiers, each one able to recognize a single dimension of the Lovehim’s cube.

Table VIII shows a comparison of the results obtained through a flat classification and a classification based on Lovheim theory, respectively. The application of this model has determined a notable increase in accuracy. In particular: there is an increment, of precision (14.3%), of recall (11.6%), of precision (12.9%). The applied assumption determined a little better results: there is an increment of precision (0.7%), of recall (2.4%), of precision (0.2%). Though the emotions of the dataset were only 7 of the 8 proposed by Lovheim, it can be asserted that Lovheim’s Cube Theory is more effective than a simple flat classification regarding the sentiment analysis.

The realization of the system on an actor framework has allowed creating a flexible architecture of composable classifiers. Moreover, the actor model allows using the realized system for online classification and continuous learning, since it naturally leads to the management of streams of asynchronous messages. Neurotransmitter (assumption) (%) P R F1 56.8 65.9 61.0 64.5 36.4 46.6 74.8 9.9 46.6 49.0 33.4 39.8 4.0 7.0 5.1 52.4 79.1 63.0 68.5 42.4 52.4 59.2 55.9 53.7 55.9

Neurotransmitters Confusion Matrix (assumption) Excitement Surprise Anger Disgust Fear 199 0 4 14 1460 944 6 158 4 553 157 160 117 105 448 37 15 397 1 652 0 1 0 11 78 83 3 66 9 5881 43 29 68 132 2160

R 65.9 36.5 8.7 33.9 7.0 79.2 42.6 55.7 55.7

F1 61.0 46.6 15.9 41.1 5.1 63.0 52.8 53.5

References

[1] E. Cambria , “ Affective computing and sentiment analysis , ” IEEE Intelligent Systems , vol. 31 , no. 2 , pp. 102 - 107 , 2016 .

[2] P. Fornacciari , M. Mordonini , and M. Tomaiuolo , “ Social network and sentiment analysis on twitter: Towards a combined approach .” in KDWeb, 2015 , pp. 53 - 64 .

[3] G. Lombardo , A. Ferrari , P. Fornacciari , M. Mordonini , L. Sani , and M. Tomaiuolo , “ Dynamics of emotions and relations in a facebook group of patients with hidradenitis suppurativa,” in International Conference on Smart Objects and Technologies for Social Good . Springer, 2017 , pp. 269 - 278 .

[4] P. Fornacciari , M. Mordonini , A. Poggi , L. Sani , and M. Tomaiuolo , “ A holistic system for troll detection on twitter,” Computers in Human Behavior , vol. 89 , pp. 258 - 268 , 2018 .

[5] R. J. Dolan , “Emotion, cognition, and behavior,” science, vol. 298 , no. 5596 , pp. 1191 - 1194 , 2002 .

[6] P. A. Kragel and K. S. LaBar , “ Decoding the nature of emotion in the brain,” Trends in cognitive sciences , vol. 20 , no. 6 , pp. 444 - 455 , 2016 .

[7] P. Ekman , “ Facial expression and emotion.” American psychologist , vol. 48 , no. 4 , p. 384 , 1993 .

[8] S. S. Tomkins , “Affect theory,” Approaches to emotion, vol. 163 , no. 163 - 195 , 1984 .

[9] J. Posner , J. A. Russell , and B. S. Peterson , “ The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology,” Development and psychopathology , vol. 17 , no. 3 , pp. 715 - 734 , 2005 .

[10] H. Lövheim , “ A new three-dimensional model for emotions and monoamine neurotransmitters,” Medical hypotheses , vol. 78 , no. 2 , pp. 341 - 348 , 2012 .

[11] J. Vallverdú , M. Talanov , S. Distefano , M. Mazzara , A. Tchitchigin , and I. Nurgaliev , “ A cognitive architecture for the implementation of emotions in computing systems,” Biologically Inspired Cognitive Architectures , vol. 15 , pp. 34 - 40 , 2016 .

[12] B. D. Moyle , C.-l. Moyle, A. Bec , and N. Scott , “ The next frontier in tourism emotion research ,” Current Issues in Tourism, pp. 1 - 7 , 2017 .

[13] G. Lombardo , P. Fornacciari , M. Mordonini , M. Tomaiuolo , and A. Poggi , “ A multi-agent architecture for data analysis , ” Future Internet , vol. 11 , no. 2 , p. 49 , 2019 .

[14] G. Angiani , S. Cagnoni , N. Chuzhikova , P. Fornacciari , M. Mordonini , and M. Tomaiuolo , “ Flat and hierarchical classifiers for detecting emotion in tweets , ” in Conference of the Italian Association for Artificial Intelligence . Springer, 2016 , pp. 51 - 64 .

[15] M. A. Riordan and L. A. Trichtinger , “ Overconfidence at the keyboard: Confidence and accuracy in interpreting affect in e-mail exchanges , ” Human Communication Research , vol. 43 , no. 1 , pp. 1 - 24 , 2017 .

[16] G. Angiani , P. Fornacciari , M. Mordonini , M. Tomaiuolo , and E. Iotti, “ Models of participation in social networks,” in Social Media Performance Evaluation and Success Measurements . IGI Global , 2017 , pp. 196 - 224 .

[17] A. Z. Syed et al., “ Applying sentiment and emotion analysis on brand tweets for digital marketing ,” in 2015 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT) . IEEE, 2015 , pp. 1 - 6 .

[18] P. Fornacciari , M. Mordonini , and M. Tomaiuolo , “ A case-study for sentiment analysis on twitter .” in WOA , 2015 , pp. 53 - 58 .

[19] N. Öztürk and S. Ayvaz , “ Sentiment analysis on twitter: A text mining approach to the syrian refugee crisis,” Telematics and Informatics , vol. 35 , no. 1 , pp. 136 - 147 , 2018 .

[20] S. Rosenthal , P. Nakov , S. Kiritchenko , S. Mohammad , A. Ritter , and V. Stoyanov , “Semeval-2015 task 10: Sentiment analysis in twitter ,” in Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015 ), 2015 , pp. 451 - 463 .

[21] S. Mohammad , F. Bravo-Marquez , M. Salameh , and S. Kiritchenko , “Semeval-2018 task 1: Affect in tweets,” in Proceedings of The 12th International Workshop on Semantic Evaluation , 2018 , pp. 1 - 17 .

[22] A. Yadollahi , A. G. Shahraki , and O. R. Zaiane , “ Current state of text sentiment analysis from opinion to emotion mining,” ACM Computing Surveys (CSUR) , vol. 50 , no. 2 , p. 25 , 2017 .

[23] L. Zhang , S. Wang , and B. Liu , “ Deep learning for sentiment analysis: A survey,” Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery , vol. 8 , no. 4 , p. e1253 , 2018 .

[24] G. Angiani , L. Ferrari , T. Fontanini , P. Fornacciari , E. Iotti , F. Magliani , and S. Manicardi , “ A comparison between preprocessing techniques for sentiment analysis in twitter .” in KDWeb, 2016 .

[25] A. Seyeditabari , N. Tabari , and W. Zadrozny, “ Emotion detection in text: a review,” arXiv preprint arXiv: 1806 .00674, 2018 .

[26] K. Sailunaz , M. Dhaliwal , J. Rokne , and R. Alhajj , “ Emotion detection from text and speech: a survey,” Social Network Analysis and Mining , vol. 8 , no. 1 , p. 28 , 2018 .

[27] V. K. Jain , S. Kumar , and S. L. Fernandes , “ Extraction of emotions from multilingual text using intelligent text processing and computational linguistics , ” Journal of computational science , vol. 21 , pp. 316 - 326 , 2017 .

[28] I. Perikos and I. Hatzilygeroudis , “ Recognizing emotions in text using ensemble of classifiers,” Engineering Applications of Artificial Intelligence , vol. 51 , pp. 191 - 201 , 2016 .

[29] W. G. Parrott , Emotions in social psychology: Essential readings . Psychology Press, 2001 .

[30] R. Plutchik , “Emotion,” A psychoevolutionary synthesis, 1980 .

[31] F. Bergenti , A. Poggi , and M. Tomaiuolo , “ An actor based software framework for scalable applications , ” Lecture Notes in Computer Science (LNCS) , vol. 8729 , pp. 26 - 35 , 2015 , proc. 7th International Conference on Internet and Distributed Computing Systems (IDCS 2014 ); Calabria; Italy; 2014 -09-22/24 [MT].

[32] S. Gallardo-Vera and E. Nava-Lara , “ Developing collaborative applications with actors , ” in Proceedings of the World Congress on Engineering and Computer Science , vol. 1 , 2015 , pp. 1 - 5 .