Sentiment analysis has recently gained popularity due to the emergence of social media and the semantic web, which provide platforms for individuals to express their thoughts and emotions. In the field of health, analyzing this type of data can have numerous benefits, particularly in psychology, where it can aid in automatically identifying a patient's mental state. In this study, we present a method for categorizing emotions from text data in the health field. We evaluated various classifiers such as: Convolutional Neural Networks (CNN), Gated Recurrent Units (GRU), and Zero-Shot learning (ZSL) on the unbalanced EmoHD dataset. Our findings showed that the CNN model gives the best performance[1].
Souaad HAMZA-CHERIF1, Nesma SETTOUTI1,2
Sentiments are complex phenomena that can take many forms, such as emotions, reactions to events, or mental states. They can be positive or negative, joyful or boring, sad, etc.
Sentiment analysis has become a rapidly growing field, particularly with the rise of social media and the semantic web, which make it easier for people to express their thoughts and emotions through text, emoticons, and other means.
Machine understanding and analysis of sentiments is particularly important in the field of health, where a patient’s emotional state can greatly impact their healing process. For example, a patient sufering from depression may not be in a favorable mental state to begin their recovery. Similarly, in the field of psychology, automatic sentiment analysis can be very beneficial in identifying a patient’s psychological state as well as the diferent human personality traits that can explain human reasoning and behavior.
In this context, artificial learning techniques and natural language processing (NLP) ofer promising tools which have proven themselves to provide automatic solutions in diferent fields such as health because they play an important role in understanding the meaning of content for classifying, analyzing, and predicting human sentiments by machine. In this article, we present an approach for sentiment analysis from text data in the field of health. We compare various machine learning methods, including convolutional neural networks (CNN), gated recurrent units (GRU), and zero-shot learning (ZSL).
The rest of the article is structured as follows. In section 2, we will discuss related work in the ifeld, then we will present our proposed approach and research strategy in section 3. In section 4, we will analyze the results we obtained and conclude with perspectives and future works.
Several recent studies have focused on using artificial learning for sentiment analysis, one
notable example is in [
In [
Other studies have also applied sentiment analysis to health-related textual data. In [
In [
In [
In [
In this work, we compare diferent approaches for classifying sentiments from health-related textual data (EmoHD) using CNN, GRU, and ZSL models. Our primary goal is to explore other deep learning models on the EmoHD dataset and evaluate their performance especially since these models have demonstrated their efectiveness in the field. Then, we aim to evaluate the annotation of EmoHD dataset using the Zero-Shot Learning model.
In this article, we present a method for classifying sentiments from text data in the EmoHD
[
To classify the text data in the EmoHD database, it is necessary to perform a series of preprocessing steps to remove any noise present in the data and improve the classification accuracy. The main steps included are: • Lowercase conversion:
Converting all terms to lowercase is necessary to avoid duplicates, as even though the meaning of words such as "Health" and "health" are the same, they are treated as separate lexical units if they are in diferent cases. • Removal of punctuation:
Removing all punctuations from the text that do not provide any useful information is a process that helps to improve the performance of data classification. • Remove stop words: These are words that are very common in the language being studied but do not provide any informative value for understanding the "meaning" of a document or corpus, so they are removed. • Removal of rare and common words:
To avoid the negative impact that rare and common words can have on the classification, we remove them by counting their frequency of appearance in the text, this helps in reducing the noise generated by them in the text. • Lemmatization:
It consists of replacing each word with its canonical form, for example, the word "known" is replaced with its canonical form "know". This step is useful for the thematic classification of texts because it treats diferent variants resulting from the same form or root as a single word. • Tokenization:
It’s the act of parsing text into tokens, in other words, the text is segmented into linguistic units such as words, punctuations, numbers, alphanumeric data... Each element corresponding to a token which will be useful for the to analyse.
The EmoHD dataset has six class labels: Angry (1343), Excited (1215), Fear (742), Happy (522), Sad (358), and Bored (22). As the data is unbalanced, it is necessary to perform resampling to balance the data. Our approach involves using naive oversampling by randomly duplicating observations from the minority class in order to increase their representation. This is done by re-sampling with replacement.
• Convolution Neural Network (CNN)
Convolutional Neural Networks (CNNs) are types of deep, feedforward artificial neural
networks composed of layers of nodes, including an input layer, one or more hidden
layers, and an output layer [
Next, we have compile our model. Compiling the model takes three parameters: optimizer, loss and metrics. The optimizer controls the learning rate. We will be using ‘adam’ as our optmizer. Adam is generally a good optimizer to use for many cases. The adam optimizer adjusts the learning rate throughout training. The learning rate determines how fast the optimal weights for the model are calculated. We used ‘categorical_crossentropy’ for our loss function. This is the most common choice for classification. A lower score indicates that the model is performing better.And we used the ‘accuracy’ metric to see the accuracy score on the validation set when we train the model. • Gated Recurring Unit (GRU)
GRU (Gated Recurrent Unit) is an improved version of the standard recurrent neural network that was introduced in [11]. It has some advantages over long-term memory (LSTM) in certain cases, such as using less memory and being faster.
GRUs solve the leakage gradient problem of a standard RNN by using two gates, the update gate and the reset gate. These gates are two vectors that decide what information is allowed to pass to the output and can be trained to retain information from far back in time. This allows for relevant information to be passed along a chain of events for better predictions.
The GRU model implemented in our approach has 3 layers: an embedding layer (defined in the previous section), a GRU layer (this layer is a fully connected layer with Gated recurrent units instead of simple neurons (we used 64 GRU), which are an improved version of standard recurrent neural network. The GRU is similar to a long short-term memory (LSTM) but with fewer parameters), and a dense layer with a softmax function activation.
Next, we compiled our model with the three parameters: optimizer, loss, and metric (previously quoted in the CNN model). • Zero Shot Learning (ZSL)
Zero-shot learning (ZSL) is the ability to perform a task without any prior training examples. It is used to build models for classes that have not yet been labeled. ZSL transfers knowledge from known classes to new classes using class attributes as the basis of this transfer. The process of ZSL has two phases: – Training: Gaining knowledge about the attributes – Inference: Utilizing the knowledge to classify examples into new classes. In our approach, we implement zero-shot classification by using the transformer library on the EmoHD dataset. Our proposed model takes the candidate labels ("Angry", "Bored", "Happy", "Sad", "Excited", "Fear") from the EmoHD dataset and the text vectorized by the sentence transformer. The output of the zero-shot model is a list of emotion labels mapped to their probabilities for the given input text.
In this research, we evaluated the performance of GRU, CNN, and Zero-Shot Learning models on the EmoHD dataset. Additionally, we analyzed the labels of the EmoHD dataset using Zero-Shot Learning. For our testing procedure, we split the data into a training set (80%) and a validation set (20%).
To evaluate the performance of each model, we used the F1-score, which is a commonly used evaluation metric that is more appropriate for datasets with imbalanced classes than accuracy. The F1-score is a combination of precision and recall, and takes into account both false positives and false negatives. The formula for the F1-score is represented in equation 1. 1 − = 2 × (2 × + + ) (1) We trained various machine learning algorithms as described in the "Classification models" section and evaluated their performance on the EmoHD dataset on 12 epochs. In our study, we evaluated the classification models with and without re-sampling the data. The results of these evaluations are presented in Table 1.
From the results in table 1, it can be seen that the CNN model achieved the highest score of 82% while the GRU model had a score of 80% after re-sampling the data. Thus, we concluded that re-sampling improves the classification performance.
We also noticed poor performance of the Zero-shot Learning model. To understand these results, we randomly selected 50 text samples from the EmoHD dataset and classified them according to the output class labels (Angry, Excited, Happy, Fear, Sad, Bored) to compare the results of the ZSL model with the existing labels in EmoHD. An example of this can be seen in Table 2.
From the 50 samples, only 12 predictions matched the labels of EmoHD. We also observed that the confusing labels were particularly around the "Excited" label which was frequently confused with other labels.
In this study, we proposed a method for sentiment analysis using unbalanced textual health data. Our research focused on examining the efect of unbalanced data on text classification and found that re-sampling improves classification performance. Additionally, we found that the CNN model did not perform better than other models in terms of F1-score. Furthermore, we karachi dengue fever claim another life frontier post may fp report karachi woman died dengue fever private hospital karachi total number death fatal fever karachi reached two according detail deceased woman identified balqees bin qasim locality karachi toll infected patient city reached since beginning year according dengue surveillance cell total number dengue case far reported karachi month may.
Score label ZSL
EmoHD label 0.278:(Sad) 0.270:(Fear) 0.188:(Excited) 0.096:(Angry) 0.095:(Bored) 0.070:(Happy)
Fear discovered that the similarities in semantics among the labels used in the EmoHD dataset led to confusion, particularly with the term "Excited" which can be mistaken as both "angry" and "happy". As next steps, we plan to continue our experiments in sentiment analysis and explore the possibility of creating new textual databases from web data. [10] S. G. Tesfagergish, J. Kapociute-Dzikiene, R. Damasvicius, Zero-shot emotion detection for semi-supervised sentiment analysis using sentence transformers and ensemble learning, Applied Sciences 12 (2022). URL: https://www.mdpi.com/2076-3417/12/17/8662. doi:10.3390/app12178662. [11] J. Chung, Ç. Gülçehre, K. Cho, Y. Bengio, Empirical evaluation of gated recurrent neural networks on sequence modeling, CoRR abs/1412.3555 (2014). URL: http://arxiv.org/abs/ 1412.3555. arXiv:1412.3555.