An Approach for Prediction of User Emotions Based on ANFIS in Social Networks

Abstract

In this paper, we propose an approach for emotions prediction. We suggest a taxonomy-based detection of user joyful interests with semantic spaces and also we propose an ANFIS method for prediction of emotions used in Twitter posts. Catching the attention of a new acquaintance and empathize with her can improve the social skills of a robot. For this reason, we illustrate here the first step towards a system which can be used by a social robot in order to “break the ice” with a new acquaintance.

1. Introduction

One of the most relevant steps in making new acquaintances in the ``engagement'' phase, which is a very complicated phenomenon involving both cognitive and affective components, including attention and enjoyment [ 1 ] [ 6 ]. For this reason, there has been a growing interest about this specific phase in the human-machine-interaction (HMI) field [ 2 ].

With the term ``engagement'' we refer to ``starting or intention to start an interaction''. In particular, we believe that in making new acquaintances ``first impressions are everything''. For this reason, finding common interests to ``talk about'', can make it possible to start an empathetic interaction between an human and a robot, improving the human-machine interaction effectiveness.

In order to trigger both attention and enjoyment, given these premises, it could be useful to design a social robotic system which tries to discover topics that can be interesting for the just met interlocutor, attempting to understand what might raise a sentiment of joy in order to to catch an empathetic attention of the user.

As a matter of fact, the knowledge of the topics of interest and the ``joyful’’ subjects for the user can lead the first stages of a conversational interaction that allows the robot to ease the engagement phase, instead of a standard and overly prepared interaction between a robot and an human user.

To achieve this objective, the robot can be able to access the social network posts of the new acquaintance trying somehow to detect her/his interests, which let arise a joyful feeling in her/him to start an, hopefully, interesting conversation for the user [ 3 ].

Social networks represent maybe the best place to gather information about people’s opinions, as a matter of fact, social media users generally express personal thoughts and to discuss with others about specific subjects [ 4 ] [ 19 ]. These opinions are actually valuable to understand and classify the emotion of an event, a product, a person, etc. and analyze his trend [ 13 ] [ 14 ] [ 5 ].

During the last decade the use of emoji has increasingly pervaded Social Media platforms by providing users with a rich set of pictograms useful to visually complement and enrich the expressiveness of short text messages. Nowadays this novel, visual way of communication represents a de-facto standard in a wide range of Social Media platforms including fully-fledged portals for usergenerated contents like Twitter, Facebook and Instagram as well as instant-messaging services like WhatsApp. As a consequence, the possibility to effectively interpret and model the semantics of emojis has become an essential task to deal with when we analyze Social Media contents.

Even if over the last few years the study of this new form of language has been focusing a growing attention, at present, the body of investigations that deal with emojis is still scarce, especially when we consider their characterization from a Natural Language Processing (NLP) standpoint.

In general, exciting and highly relevant avenues for research are still to explore with respect to emoji understanding, since emojis represent often an essential of Social Media texts and thus ignoring or misinterpreting them may lead to misunderstandings in comprehending the intended meaning of a message. The ambiguity of emojis raises an interesting question in human-computer interaction: how can we teach an artificial agent to correctly interpret and recognise emojis' use in spontaneous conversation? The main motivation behind this question is that an AI system able to predict emojis could contribute notably to better natural language understanding and thus to different natural language processing tasks such as generating emoji-enriched social media content, enhancing emotion/sentiment analysis systems, and improving retrieval of social network material, and ultimately improving user profiling.

In this paper we illustrate the design of a system which can be used for detection emotions from social media content and prediction user emotions based on neuro-fuzzy network from tweets. The proposed system will be used for prediction of emoji based on tweets on one hand and for detection user emotion on other.

2. The System

The proposed system is composed of a set of modules interacting in order to catch the attention of the user. The system exploits a training phase, where a semantic space $S$ is induced from Twitter data and a joyful-topic-detection process, which exploits the Twitter ID of the user in order to retrieve her posts and trying to catch the interests of the user that somehow let arise a ``joy'' emotion. The topic detection is obtained by mapping the user tweets to the IAB taxonomy [ 8 ] [ 15 ]. \begin{figure}[tb] \centering \includegraphics[width=0.6\textwidth]{figures/architettura2.png} \caption{\emph{Joyful-topic-detection process}}\label{architettura} \end{figure}

2.1. The IAB Taxonomy

The Interactive Advertising Bureau (IAB) Tech Lab Content Taxonomy is a taxonomy which is also an international standard to map contextual business categories [ 8 ][ 9 ]. This taxonomy is particularly suited for being used by companies in the market, it is standardized and industry-neutral. These characteristics can be effectively exploited for profiling an user interests.

We have used this solution just for convenience, being the approach applicable on different targets.

2.2. Emotion Detection Module

This module is responsible for the detection of emotions in tweets. To perform this task, we have taken into consideration the six Ekman fundamental emotions [ 7 ]: anger, disgust, fear, joy, sadness and surprise, exploiting an emotions lexicon obtained from the Word-Net Affect Lexicon, as it has been illustrated in [ 17 ] [ 18 ]. The module exploits a methodology that has been described in [ 15 ][ 3 ]. The technique is based on the Latent Semantic Analysis paradigm (LSA), a methodology that is capable of giving a coarse sub-symbolic encoding of word semantics [ 12 ] and of simulating several human cognitive phenomena [ 11 ].

3. Adaptive Neuro-Fuzzy Inference System

The Adaptive Neuro-Fuzzy Inference System (ANFIS)

ANFIS is the abbreviation Adaptive Neuro-Fuzzy Inference System - an adaptive network of fuzzy output. Proposed in the early nineties [ 10 ], ANFIS is one of the first variants of hybrid neural-fuzzy networks - a neural network of direct signal propagation of a special type. The architecture of the neural-fuzzy network is isomorphic to the fuzzy knowledge base. Neuro-fuzzy networks use differentiated implementations of triangular norms (multiplication and probabilistic OR), as well as smooth functions. This allows the use of cross-fuzzy neural networks, rapid algorithms for learning neural networks, based on the method of back propagation of errors. The architecture and rules for each layer of the ANFIS network are described below. ANFIS implements the Sugeno fuzzy inference system in the form of a five-layer neural network of direct signal propagation. The system works as follows:  the first layer is the terms of the input variables;  the second layer is antecedents (parcels) of fuzzy rules;  the third layer is the normalization of the degree of implementation of the rules;  the fourth layer is the conclusion of the rules;  the fifth layer is the aggregation of the result, du according to different rules.

The network inputs in a separate layer are not allocated. Figure 1 shows an example of an ANFIS network with two input variables x1 and x2 and four fuzzy rules.

The ANFIS network functions as follows [ 16 ]: Layer 1. Each node of the first layer represents one term with a bell-like membership function.

The inputs of the network x1, x2, ..., xn are connected only with their terms. The number of nodes of the first layer is equal to the sum of the powers of the term-sets of the input variables. The output of the node is the degree of belonging of the value of the input variable to the corresponding fuzzy term.

Layer 2. The number of nodes of the second layer is m. Each node of this layer corresponds to one fuzzy rule. The node of the second layer is connected to those nodes of the first layer, which form the antecedents of the corresponding rule. Therefore, each node of the second layer can receive from 1 to n input signals. The output of the node is the degree of execution of the rule, which is calculated as the product of the input signals.

Layer 3. The number of nodes of the third layer is also equal to m. Each node in this layer calculates the relative degree of fuzzy rule execution.

Layer 4. The number of nodes of the fourth layer is also equal to m. Each node is connected to one node of the third layer, as well as to all inputs of the network (in Figure 1) links with inputs are not shown). The fourth layer node calculates the contribution of one fuzzy rule to the network output.

Layer 5. The only one node of this layer summarizes the contributions of all rules.

Typical procedures for learning neural networks can be used to configure an ANFIS network because it uses only differential functions. Usually a gradient descent combination is used in the form of an algorithm for back propagation of an error and a method of least squares. The error back propagation algorithm configures the rules of antecedents of the membership functions. The method of least squares evaluates the coefficients of the conclusions of the rules, since they are linearly related to the output of the network. Each iteration of the setup procedure is performed in two stages. At the first stage, a training sample is fed to the inputs, and the optimal parameters of the nodes of the fourth layer are found by the discrepancy between the desired and actual network behavior by the iterative least squares method. In the second stage, the residual residual is transferred from the network output to the inputs, and the parameters of the nodes of the first layer are modified by the method of back propagation of the error.

In this section we presented neuro-fuzzy network which can be used for prediction of user emotions from tweets. It is concept for the future research. We suggest to use neuro-fuzzy network for prediction of user emotions because emotions has a fuzzy nature. We put in the inputs of neural network selected keywords from tweets and in output we predict the six emotions: anger, disgust, fear, joy, sadness and surprise. In neuro-fuzzy network we can develop a fuzzy rules for better emotion prediction. In our future work we will to present working model and first results.

Acknowledgments

This work was supported by the Russian Foundation for Basic Research (Grant No. 17-07-01558).

References

[1] Brethes L , Menezes P , Lerasle F and Hayet J ( 2004 ) Face tracking and hand gesture recognition for human-robot interaction . In: IEEE international conference on robotics and automation , vol 2 . IEEE, pp 1901 - 1906

[2] Corrigan Lee J , Peters C , Küster D and Castellano G ( 2016 ) Engagement Perception and Generation for Social Robots and Virtual Agents - Toward Robotic Socially Believable Behaving Systems - Volume I , Intelligent Systems Reference Library 105 , pp 29 - 51 , Springer

[3] Cuzzocrea A and Pilato G ( 2018 ) Taxonomy-Based Detection of User Emotions for Advanced Artificial Intelligent Applications . In: de Cos Juez F. et al. (eds) Hybrid Artificial Intelligent Systems. HAIS 2018. Lecture Notes in Computer Science , vol 10870 . Springer, Cham

[4] D 'Avanzo E and Pilato G ( 2014 ) Mining social network users opinions' to aid buyers' shopping decisions . Computers in Human Behavior, Elsevier , Vol. 51 , pp 1284 - 1294

[5] D 'Avanzo E , Pilato G and Lytras M D Using twitter sentiment and emotions analysis of Google trends for decisions making . In Program , pages Vol. 51 , Issue 3. 2017 .

[6] Delaherche E , Dumas G , Nadel J and Chetouani M ( 2014 ) Automatic measure of imitation during social interaction: a behavioral and hyperscanning-eeg benchmark . Pattern Recognit Lett.

[7] Ekman P. , Friesen W V. Constants across cultures in the face and emotion ., Journal of personality and social psychology 17 : 124 , 1971 .

[8] Interactive Advertising Bureau (IAB) Contextual Taxonomy , http://www.iab.net/ Retrieved December 2017

[9] Kanagasabai R , Veeramani A , Ngan L D , Yap G E , Decraene J and Nash A S ( 2014 ). Using Semantic Technologies to Mine Customer Insights in Telecom Industry . In International Semantic Web Conference (Industry Track)

[10] Jang J-SR . ANFIS: adaptive-network-based fuzzy inference system . In IEEE transactions on systems, man, and cybernetics . pp 665 - 685 . IEEE ( 1993 ).

[11] Landauer T K and Dumais S T “ A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge”, In Psychological review , vol. 104 ( 2 ), 1990 , pp 211 - 223

[12] Landauer T K , Foltz P W and Laham D “ An introduction to latent semantic analysis”, In Discourse processes , vol. 25 , 1998 , pp 259 - 284

[13] Liu B ( 2010 ). Sentiment Analysis and Subjectivity . In N. Indurkhya, \& F. J. Damerau , Handbook of Natural Language Processing pp 627 - 665 CRC Press.

[14] Pang B , Lee L and Vaithyanathan S ( 2002 ). Thumbs up? Sentiment Classification using Machine Learning Techniques . In Proceedings of the ACL-02 conference on Empirical methods in natural language processing - Volume 10 pp 79 - 86 . Association for Computational Linguistics.

[15] G. Pilato , E. D'Avanzo - Data-driven Social Mood Analysis through the Conceptualization of Emotional Fingerprints - Procedia Computer Science , 2018 , (in press)

[16] Averkin , Alexey N and Yarushev, Sergey.: Hybrid approach for time series forecasting based on ANFIS and Fuzzy Cognitive Maps . In: Soft Computing and Measurements (SCM) , - 2017 XX IEEE International Conference . pp 379 - 381 . IEEE ( 2017 )

[17] C. Strapparava and R Mihalcea . Semeval-2007 task 14: Affective text . In Proceedings of the 4th International Workshop on Semantic Evaluations , pages 70 - 74 . Association for Computational Linguistics, 2007 .

[18] C. Strapparava and R Mihalcea . Learning to identify emotions in text . In SAC '08 Proceedings of the 2008 ACM symposium on Applied computing . 2008 .

[19] Terrana D , Augello A , Pilato ( 2014 ). Facebook users relationships analysis based on sentiment classification . Proc. of 2014 IEEE International Conference on Seman- tic Computing (ICSC) , pp 290 - 296