EEG based emotion recognition of image stimuli

Abstract

Introduction: Emotion is playing a great role in our daily lives. Traditional ap proaches of emotion recognition are performed based on the features including facial images, measurements of heart rates, blood pressure, temperatures, tones of voice/ speech, etc. However, these features can potentially be changed to fake features by a person while it is recorded. To unhide the real features, the brain signal representing the response of a person is recorded directly from his/her brain. The various ways of measuring brain waves: electroencephalogram(EEG), Magnetoencephalography(MEG), functional magnetic resonance imaging (FMRI), etc. For this study, EEG is chosen for emotion recognition relying on cost effectiveness and performance trade-offs. Brain computer interface(BCI) based emotion recognition is used in a vari ety of applications including advertisement, patient treatment, depression management, music player, human computer interaction, detecting children learning disabilities, assistive technologies, game playing, automatic addition of emotional pictures during conversation, emotion enabled avatar, neuromarketing ,etc.[1]. The brainwave activity is broadly divided into five frequency bands. The boundary between the frequency bands is not strict but does not vary much. The frequency bands include delta(0.5-4Hz), theta(5-8Hz), alpha(9-12Hz), beta(13-30Hz) and gamma(above 30Hz) [2,3]. The main purpose of this study is to detect emotion based on EEG signal analy sis recorded from brain in response to visual stimuli. Numerous studies demonstrated that even though it is possible to measure emotion from EEG signals recorded from stimulated brain in practice, the outputs of Brain Computer Interface(BCI) related re search works are different with same stimuli and with brain response of same or dif ferent subjects [4]. The other problem is that parts of the brain that responds to emo tion is not clearly identified or/and mixed up in research results. For example, emotion is responded either or both on frontal lobe or temporal lobe. Besides this, the brain wave containing emotion is not clearly understood to be in alpha frequency band or gamma frequency band. This problem inspires us to work on it. This study attempts to find out answers for the following research questions: (1) What regions of the brain are associated with visual emotion? (2) Which frequency bands of the brain waves are used for emotion recognition? (3) How accurately the chosen features were recognizing emotions using machine learning approaches? Methods: We collected three image data sets include: 90 sample images of Geneva Affective Picture Database (GAPED), 8 colour images and 36 Indian company logo. EMOTIV EPOC head sets, Emotiv EPOC TestBench Control panel software and EventIDE are used for EEG brain activity recording. The Spectrum Density (PSD) of

-

all 14 channels along with five frequency bands including: theta, alpha, low_beta, high_beta and gamma frequency bands are measured. A total of 70 channels are recorded for each of 11 subjects(person) samples for each of the three image data sets. These bands are filtered. The proposed approach consists of six stages: image stimulus presentation, subjects, EEG signal recordings, signal filtering, feature extraction and classification. See Fig. 1 of Appendix. For pre-processing and building machine learning models, weka software environment is used.

Results: To answer research question one and two, the top ranked features for each subject are extracted using Relief algorithm in [ 5 ]. The brain frequency bands where it has top ranked features are counted for each subject. On the basis of this, 37.5% of the subjects responded to emotional images with alpha brain waves. Alpha waves is dominant feature which carries the most of all brain waves. The brain frequency bands and the channel numbers are counted in each of the three experiments on the three data sets. With similar approach, we empirically observed that the frontal lobe of human brain is more responsible for emotional function than other parts of cerebral cortex. We split the filtered data into train and test sets. We trained the most popular supervised machine learning (Bayesian Network, J48(decision tree), Adaboost(meta learner) and Random forest). After fine tuning the parameters of the selected machine learning models, the performances of these models are tested on test sets. The inten tion is to use these machine learning models to predict the emotion type (positive/negative) in response to the presented stimuli using test sets. The performance of these models are tested. For the three data sets, the average accuracy of machine learning algorithms (i.e. J48, Bayes Net, Adaboost and Random Forest) is 78.86, 74.76, 77.82 and 82.46, respectively. For more detail, see Table 1 of Appendix.

Conclusions: For feature selection, we used relief algorithm along with ranking algorithm. Because this feature evaluation algorithm is more appropriate for recognizing relevant features from irrelevant feature that may cause degrading the performance of machine learning algorithm. In this study, we tried to address three key issues. First, we empirically identified the brain regions which are more responsible for emotion. On the basis of feature evaluation result, frontal lobe is more emotionally informative than other regions of the brain. Second, alpha frequency bands are more discriminator than other brain frequency waves for emotion recognition.

In addition, this study attempts to develop an EEG based Emotion recognition of image stimuli. It is constrained by a number of challenges dominantly lack of quality in the recording of EEG data. The level of attention of subjects, the variability arise in multiple session, the variability caused by muscle movement, the variability due to machine noise, differing physiology of subjects, differ cognitive patterns and differing behavior of subjects [ 4 ]. We achieved encouraging results. The EEG based approach is tested for logo color preference detection in the context of neuromarketing. The re sults empirically demonstrated detection of the favorite color preferences of the customers in response to the logo color of an organization or service. For interested re searchers, we recommend to work on EEG based researches in the areas of neuromarketing, TV ads evaluation, product branding, product preferences, disability treat ment, stress management, just to name a few.

References

1. Y. Liu , O. Sourina , and M. K. Nguyen , “ Real-time eeg-based emotion recognition and its applications,” in Transactions on computational science XII , pp. 256 - 277 , Springer, 2011 .

2. M. Hedda Šola , “ Neuromarketing-science and practice,” FIP-Financije i pravo , vol. 1 , no. 1 , pp. 25 - 34 , 2013 .

3. “Neuromarketing india.” http://neuromarketing-india. blogspot.com/ , 2014 .

4. J. M. Williams , “ Deep learning and transfer learning in the classification of eeg signals ,” 2017 .

5. S. F. Rosario , K. Thangadurai , et al., “Relief: feature selection approach ,” International journal of innovative research and development , vol. 4 , no. 11 , pp. 218 - 224 , 2015 . Fig. 1 Proposed System Table 1. Performance of the machine learning models for the three data sets .