Sentiment analysis is one of the most researched topics in the computer science domain. Whenever the term opinion appears, sentiment analysis is required. Many business sectors are growing by analyzing users opinions about their products. E-commerce portals like Amazon and Flipkart ofering users to express their opinion by posting the purchased product review. Further, the next buyer of the same product utilizes the user's review to make their decision-should purchase or not. Existing models of sentiment analysis mostly referred to English language textual comments. However, currently, users are posting the comments and reviews in mixed languages like Hindi-English, Malayalam-English and similar ones; it is called code-mixed languages. To identify the user sentiment from the code-mixed language, this research suggested a deep learning-based framework. The proposed framework automatically extracts the features from input sentences and predicts their sentiment with a 0.552 F1-score for the best case.
People are expressing their opinion about things using natural language on diferent platforms,
including YouTube, Facebook, Twitter and others [1, 2]. Analyzing the user’s post and finding
its opinion plays a vital role in the decision-making system and has the power to lift or down
accordingly. For example, an E-commerce portal like Flipkart ofering users to express their
opinion about the product in the form of a review. This review helps the buyer to take their
decision like whether the product is good or not [
To process the comments or user’s review, many frameworks developed earlier using various
machine learning and deep learning frameworks [
The research community has recently been interested in sentiment analysis of code-mixed
language. Kumar et al. [
In line with the works developed for sentiment analysis from code-mixed social media posts,
we developed a deep neural model using Bi-directional LSTM [
The rest of the paper is organized as follows: Section 2 discusses the proposed methodology. In Section 3, we discuss the experimental outcomes, and finally, Section 4 concludes the work.
This research suggested a framework to predict the sentiment analysis of code-mixed data using
Bi-directional LSTM model Neural Network [
The original dataset contains many unsupportive characters, which needs to the filter out before passing it to the model for processing. The data cleaning step is performed to remove the emojis, special characters, non-ASCII characters. The number is removed and converted all information into lower cases. Further, the cleaned data passes are padded with zeros for making all messages of equal length. The maximum size of the message is fixed to 30 and 70, respectively, for the word and char level processing. The padded text passes to the embedding layer, where for each word, their corresponding vector is extracted from a pre-trained word embedding called Glove[17]. The GloVe embedding, having the dimension of 100, means each 1https://dravidian-codemix.github.io/2021/datasets.html
Code-Mixed Dataset
Preprocessing
Tokenization Char Level Padding (70)
Word Level
Padding (30) Random Embedding (100)
Random Embedding (100)
Pre-trained Word
Embedding (100) Bi-LSTM (64)
Dropout
Bi-LSTM(32)
Bi-LSTM(32) Dropout
Dropout Concatenate
Dense(128)
Output Layer, Dense(5) input word is mapped into 100-dimensional vectors. This way, for each message of size (n) and × 100 sizes matrix created by the embedding layer. Random embedding technique is also used for the word and char level dataset as shown in Figure 1 with the same output dimension as 100.
Further, the embedded dataset passes to Bi-directional Long-Short Term Memory (Bi-LSTM) model for further processing. For Char embedding, 64 units of Bi-LSTM were used, whereas for processing the words, 32 units of Bi-LSTM. The dropout layer is added in all three cases, and then the outcomes are concatenated together. The concatenated outcomes of the Bi-LSTM models are passes to a fully connected dense layer with 128 neurons, followed by an output layer consisting of five neurons. The ReLU activation function is used in the internal layer of the network; however, at the output layer, Softmax is used.
This research developed a model to classify the code-mixed input sentence in one of the
predefined sentimental categories. The evaluate the model performance, the classification metrics
called precision (P), recall (R), and F1-score (F1) are used. Precision is defined as the number
of correctly predicted sentiment categories among the retrieved instances of the particular
sentiment category. The recall is defined as the number of correctly predicted sentiment
categories among the total number of instances of that particular sentiment category. The
F1-score (F1) is the harmonic mean of the precision and recall [
A number of the experiment was done with by extracting the various n-gram features from the text using tf-idf vectorization technique and passing it to traditional Machine Learning based classifiers like- Random Forest (RF), Logistic Regression (RF), and Naive Bayes (NB). The best outcomes of these classifiers are shown in Table 2. Most of the instances are miss-classified to another category of sentiment. The positive sentiment category is predicted with the highest prediction accuracy by all three classifiers, NB, LR, and RF. In contrast, the same classifiers are failed to detect the not-Tamil sentiment category. None of the classifier’s performance was satisfactory for predicting code-mixed data of negative, mixed-feelings, unknown state and not-Tamil categories with bi-gram features.
To improve the model performance, we have used deep learning-based Bi- directional LSTM. The outcomes of the proposed B-LSTM model with validation dataset is shown in Table 4. The best performance is achieved for the Positive sentiment class. The precision, recall and F1-score values are 0.68, 0.80, and 0.74, respectively, whereas the lowest precision, recall and F1-score values are 0.20 0.11, 0.14, respectively, for mixed-feelings sentiment class. The performance of the proposed deep learning model outperforms the traditional machine learning models (Table 2) by achieving better performance for all classes. The non-Tamil classes are not recognized at all by any of the mentioned traditional ML models. However, the proposed deep learning model provides satisfactory prediction accuracy. The weighted average precision, recall and F1-score values are 0.54, 0.57, and 0.55, respectively on the validation dataset. However, on the test dataset, the weighted precision, recall and F1-score values are 0.544, 0.566, and 0.552, respectively.
One of the possible reasons behind the biased performance of the model for diferent sentimental classes may include the inconsistent distribution of data samples in diferent classes of training and testing set (Table 1). The number of samples present in the Positive sentiment category is highest, whereas the lowest number of samples is present in not-Tamil category. The efect of this data distribution is seen on the model’s outcomes (Table 4). Hence, to get better outcomes, data oversampling techniques such as SMOTE or ADASYN may help [19]. Another possible reason behind the low performance of the model may include the high number of code-mixed samples in training and testing dataset. By normalizing the dataset into English may help to achieve better prediction accuracy.
Sentiment analysis is one of the major research areas in computer science, where the opinion will be extracted from the input text. The opinion may be positive, negative or neutral. In the current time, users are popularly used mixed languages to post comments or reviews. Hence, getting the opinion from such a post is a challenging task. This research suggested a deep learning-based automated system to predict the sentiment of the user’s post written in Tamil code-mixed. The proposed framework utilised the pre-trained word embedding technique and achieved a weighted F1-score of 0.552 for the best case on test sample. [1] B. Liu, et al., Sentiment analysis and subjectivity., Handbook of natural language processing 2 (2010) 627–666. [2] R. Feldman, Techniques and applications for sentiment analysis, Communications of the
ACM 56 (2013) 82–89. of the Sentiment Analysis of Dravidian Languages in Code-Mixed Text, in: Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation, CEUR, 2021. [15] B. R. Chakravarthi, V. Muralidaran, R. Priyadharshini, J. P. McCrae, Corpus creation for sentiment analysis in code-mixed tamil-english text, arXiv preprint arXiv:2006.00206 (2020). [16] R. Priyadharshini, B. R. Chakravarthi, S. Thavareesan, D. Chinnappa, T. Durairaj, E. Sherly, Overview of the dravidiancodemix 2021 shared task on sentiment detection in tamil, malayalam, and kannada, in: Forum for Information Retrieval Evaluation, FIRE 2021, Association for Computing Machinery, 2021. [17] J. Pennington, R. Socher, C. D. Manning, Glove: Global vectors for word representation, in: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), 2014, pp. 1532–1543. [18] P. K. Roy, Multilayer convolutional neural network to filter low quality content from quora, Neural Processing Letters 52 (2020) 805–821. [19] P. K. Roy, Z. Ahmad, J. P. Singh, M. A. A. Alryalat, N. P. Rana, Y. K. Dwivedi, Finding and ranking high-quality answers in community question answering sites, Global Journal of Flexible Systems Management 19 (2018) 53–68.