Nowadays, the use of emojis is very common to show our emotions with just a single image instead of long sentences describing our emotions. Each emoji describes a particular emotion, such as anger, disgust, fear, sadness, surprise, and happiness. Now if we are given a task to identify emotions in a text, that means we have to tag a text with multiple emojis, each pointing to a diferent emotion. This paper aims to check for multiple emotions in an Urdu text, which comes under the category of multi-label classification. We have used pre-trained BERT models to add basic knowledge about a language (Urdu in our case). Over the pre-trained model, we added the classification layer using PyTorch. The output layer has seven nodes, six of which are for six emotions, and the seventh is for neutral. FIRE 2022 provided the Urdu tweet dataset used here as part of the subtask ”Multi-label emotion classification in Urdu” of the main task ”Emothreat: Emotion and Threat detection in Urdu.” With the vast-scale expansion of social media, it is afecting the narrative of the whole country or even the whole world, which could be evidenced by the examples of various country-wide or worldwide campaigns started from social media accounts and spread into the population. The messages or tweets posted by various users online are responsible for all these new efects of social media, so these messages or tweets must be analysed to understand the mindset of the users about diferent topics in the public domain. What is better than an emotional classification of text is to categorise it into multiple emotions like anger, fear, sadness, etc. Emotion classification will help in identifying the mood of the population about a topic. This type of task comes under ”afective computing,” as it was defined in [ ”computing that relates to, arises from, or influences emotions,” or it can be said that ”afective
computing” is computing that has to do with emotions. [
The following sections are included in the paper: Section 2 describes the related work; Section 3 describes the dataset and the challenges that go along with it and their solutions; Section 4 describes our model design and techniques; and Section 5 describes the evaluation and result of our model over the data.
Several authors have participated in the hate speech detection tasks, such as [
Another work in[14] fine tuned monolingual and multilingual transformers over Urdu text and used ensembling techniques to combine the results of RoBERTa-urdu-small, XLM-RoBERTa, bert-based-multilingual-case and Alberta-urdu-large and get the accuracy of 0.596 and F1 score of 0.449. The author of [15] got the highest F1 score of 0.7993 by using pre-trained BERT models + fine tuning classification layer over them. They also used data augmentation to make the models generalise better and used both machine learning and deep learning techniques for the task of recognising hate and ofensive speech. The efectiveness of several pre-trained multilingual BERT models in the detection of threats and hate speech, which are also types of emotions, is discussed in [14] and [15]. [16] surveys the concept of emotion detection by exploring various methods of categorising emotions one is Direct Emotion Detection, which considers 6 or more basic emotions, believes that all other emotions are a combination of these basic emotions and considers each basic emotion to be independent, whereas Dimensional Emotion Detection, which does not consider emotions to be independent, defines a 2-D or 3-D space for emotion categorisation. The X-axis represents valency, while the Y-axis represents arousal. Each area in the 2-D space shows a certain kind of emotion, and you can also add a Z-axis showing the person’s control over that emotion.
The dataset is provided by FIRE 2022, under the sub-task A(Multi-label emotion classification in Urdu) of main Task: EmoThreat: Emotion and Threat detection in Urdu. The training dataset contains sentences in Urdu and seven labels for each sentence in one-hot encoding, which is multi-label (0 or 1). the code for this task is available on this1 github repository. The distribution statistics for each label is as depicted in Table 1 for training data. Each label corresponds towards a particular emotion like anger, disgust, fear, sadness, surprise, happiness, or neutral. The pictorial depiction of training data is as in Figure 1, which on analysis brings about a problem of unbalanced data that could afect the training model parameters in a way that is biased towards labels that have a high number of sentences labelled for it. for example, in the dataset, the label ”neutral data” has a higher count than all others; this problem of unbalanced data is taken care of in further sections by using the method of calculating positive and negative weights for each label.
1https://github.com/saransh-goel/emotion . 4. Proposed Techniques and Algorithms 4.1. Multi-label classification [17]For a d-dimensional input data, ∈ and = {1, 2, ..., } set of labels where q is the number of labels. Each instance ∈ can be associated with the subset of labels ∈ 2 which are called as relevant labels for x and the set of labels in complement of L, i.e =̄ ⧵ are called as irrelevant labels for x. Training dataset for multi-label classification of size l, will be a set of elements ( 2 ).
.., {( 1, 1), ...., ( , ), ...( , )} Multi-label classification would be learning a function () ∶ → 2 methods for multi-label classification: 1. Data decomposition method 2. Algorithm extension method . There are two main
This method includes binary classifiers, One vs Rest method, One vs One method etc. The widely used trick used by this method is to define a function for each class i.e () ∶ → , = 1, 2, .., such that () > (), ≠ ∈ . which means that relevant label should have ranked higher than irrelevant labels. .., () > (), ∈ , ∈
̄ () = {, . () ≥ , = 1, 2, .., } (1) (2) Further a threshold can be set for relevant labels, now the methods like one vs rest that the proposed model used came in picture to set this threshold along with the help of binary classifiers like Naive Bayes, SVC and Logistic regression. [17]One vs Rest method divides a q-class multilabel data set into q binary subsets, here the ℎ subset consists of positive instances with the ℎ label and negative ones with the all other labels. This method helps in identifying the threshold t in eq 2.
This method includes using multi-class classifiers and dealing with multi-label classification in one function only, like in the proposed model, which uses the BERT pre-trained model with a classification head over it for the multi-label classification.
The BERT variants used as base models are UrduHack and distil-BERT, which can work with
multilingual data as the dataset contains sentences in Urdu. For multi-label classification, as
shown in figure 2, in the proposed model there are seven parallel feed forward dense layer
networks for each class. Each gives a two-node output and works as a binary classifier for its
own class.The training process begins with data tokenization and padding with the required
[CLS] and [SEP] tokens, followed by passing the tokenized text as input to the Bert model and
using the output from the Bert model corresponding to the [CLS] token as input to classification
layers as described in [
Learning Rate 1e-05 Number of Epochs 4
Batch Size 2 Figure 2: The Proposed Architecture for BERT
[
The proposed results demonstrate that the TF-IDF feature extraction model works better than the BERT model. This is because in emotion detection, keywords are found to be more important than context, as each emotion has its own set of keywords that help a lot with classification. This paper only deals with emotion detection in text data, but as explained in one of the earlier sections, a multi-modal approach to emotion detection is very efective as other features other than text, such as audio pitch and facial expression, more clearly explain an emotion. For image data, an expression detection model could help identify diferent emotions, and just as each emotion has its own set of expressions, sometimes the same sentence has diferent meanings with diferent expressions, such as ”he is very intelligent.” This sentence with a good expression will come in the category of happiness and praise, but with an expression of sarcasm, it will come under the category of jealousy. The same is true for audio, where pitch can help distinguish between anger, excitement, and a lazy tone. [12] S. Kalraa, Y. Bansala, Y. Sharmaa, Detection of abusive records by analyzing the tweets in urdu language exploring transformer based models (2021). [13] S. Kalraa, M. Agrawala, Y. Sharmaa, Detection of threat records by analyzing the tweets in urdu language exploring deep learning transformer-based models (2021). [14] S. Kalraa, P. Vermaa, Y. Sharmaa, G. S. Chauhanb, Ensembling of various transformer based models for the fake news detection task in the urdu language (2021). [15] S. Kalraa, K. N. Inania, Y. Sharmaa, G. S. Chauhanb, Applying transfer learning using bert-based models for hate speech detection (2020). [16] F. A. Acheampong, C. Wenyu, H. Nunoo-Mensah, Text-based emotion detection: Advances, challenges, and opportunities, Engineering Reports 2 (2020) e12189. [17] J. Xu, An extended one-versus-rest support vector machine for multi-label classification, Neurocomputing 74 (2011) 3114–3124.