The most serious issue with code-mixing is that people switch between languages (for example, Malayalam and English) and type in English instead of writing Malayalam words. Traditional NLP models can't handle data code-mixing data. Sentiment Analysis on Kannada-English, Malayalam-English, or Tamil-English code-mixed datasets based on five labels is the Dravidian Code-Mixed FIRE 2021 challenge. The classification is to be done based on the following labels 'Not-Malayalam, "Neutral state, 'Positive, 'Mixed feelings, 'Negative'. This paper focuses on Malayalam-English code-mixed data sentiment analysis based on the Ensemble voting model with machine learning models - Support Vector machine (SVM) and Logistic Regression and Bagging. The Hard Voting classifier model provided an accuracy : 67.78 % and F1-score : 67.53%.
same discussion is code mixing [
During sentiment analysis the sentiments are extracted from the dataset and categorize them
for usage in a variety of applications such as customer service, social media content moderating
and reputation management, [
Advani et al. proposed a machine learning algorithm that can detect the diference between
positive and negative feelings based on lexical, metadata features and sentiment[
The Dravidian Code-Mix FIRE 2021 project’s purpose is to categorize YouTube comment codemixed datasets written in Kannada-English, Malayalam-English, or Tamil-English into five labels based on sentiment polarity. Emotion polarity is assigned to each comment at the comment level. In real-world situations, this dataset has concerns with class imbalance. In Dravidian-CodeMixFIRE 2021 challenge more dataset was provided with the majority of the comments written in either Malayalam, Tamil, or Kannada grammar with English lexicon or Malayalam, Tamil, or Kannada lexicon with English grammar. Variety of comments were written in Malayalam, Tamil, and Kannada script, with English phrases interleaved. The detailed description of datasets used is provided in two papers [17] and [18] along with details on how the data was acquired and labeled in the datasets. In Table 1, the statistics for each class are tabulated.
We employed pre-processing methods to clean the comments in the dataset, such as replacing emojis with similar words, changing the comment to lower case, eliminating stop words, HTML tags, and accented characters.
Ensemble learning can take several forms, such as stacking, bagging, and so on. Because of the bagging method’s eficiency and ease, this paper created an ensemble model as both hard voting and soft voting classifier based on bagging by combining three classifiers. Conventional Machine Learning classifier; Support Vector Machine (SVM) [ 19], bagging model and Logistic Regression [20] are combined to create an ensemble model for Dravidian code mix sentiment identification. The goal of ensembling of simple classifiers is to create a robust classifier that takes advantage of each classifier’s strengths. TF-IDF [ 21] vectors obtained during feature engineering are used to train the model. Figure1 describes the model block diagram.
We used a Windows 10, 64-bit operating system, Intel Core i5 CPU 2.40 GHz, and IDE drive for code-mix data sentiment analysis. We employ Anaconda 2019.10, an open-source program for developing Machine Learning projects and current Python libraries such as scikit-learn and Pandas (IDE). To extract features, the tf-idf technique [21] is used. Various parameters were tried, but the best results were provided by the following parameters. For SVM model, radial basis function kernel (‘rgf’ )is used with a ’C’ value of 100. The ’c’ value used in logistic regression is 100, and the kernel is ‘linear.’ The bagging model used in this experiment use SVM as a base estimator. The ensemble model used for classification is majority voting. We had trained the model for Malayalam English code-mix data. Sklearn library [22] is used to build the model.
Hard Voting classifier provided an accuracy of 67.78 % and precision of 67.17% and recall of 67.78% and F1-score of 67.04%. Soft voting classifier have provided an Accuracy of 67.53% and precision of 66.93% and Recall of 67.53% and F1-score of 66.96% respectively. In Table 2 the classification metrics of Hard Voting classifier is provided and in Table 3 the classification metrics of soft ensemble model on test data is provided. The overall model performance is provide in the Table 4. From the Table 4, it is clear that Hard Voting model achieved slightly Label Positive unknown_state Negative Not-Malayalam Mixed_feelings Label Positive unknown_state Negative Not-Malayalam Mixed_feelings
This paper uses a Hard and Soft voting ensemble model to categorize Code-Mixed Malayalam and English comments dataset. Hard Voting ensemble model provided slighly better accuracy than Soft Voting classifier. In the future, applying a deep learning ensemble model on the dataset can improved the model performance and fine-tuning the model to find sarcastic Manglish comments.
The demand for sentiment analysis of text from social media, which is usually code-mixed, is growing. However, there are limited tools available to develop models specifically for codemixed data. The papers which provide details of codemix data are listed below. The research in [23] provide details of a dataset of Dravidian languages (Kannada, Malayalam, and Tamil) for detecting abusive language in social media and filtering user-generated material in local languages. In the study [24], the authors introduce a Kannada CodeMixed Dataset (KanCMD), a code mixed text, multi-task learning dataset in Kannada language for ofensive language detection and sentiment analysis. The paper [25] describe a resource named (TamilMemes) which is used to recognizing a troll meme in Tamil, which help in detecting and dealing with trolls so that before harming an individual. Chakravarthi et al. explain how they created the corpus and assigned polarities to code-mixed Malayalam-English code-mixed data[26] and Chakravarthi et al. provides details of Tamil-English [27], respectively for sentiment analysis using comments on social media, thereby providing a benchmark dataset. [10] S. Banerjee, B. R. Chakravarthi, J. P. McCrae, Comparison of pretrained embeddings to identify hate speech in indian code-mixed text, in: 2020 2nd International Conference on Advances in Computing, Communication Control and Networking (ICACCCN), IEEE, 2020, pp. 21–25. [11] S. Sharma, P. Srinivas, R. C. Balabantaray, Text normalization of code mix and sentiment analysis, in: 2015 international conference on advances in computing, communications and informatics (ICACCI), IEEE, 2015, pp. 1468–1473. [12] P. Rani, S. Suryawanshi, K. Goswami, B. R. Chakravarthi, T. Fransen, J. P. McCrae, A comparative study of diferent state-of-the-art hate speech detection methods in hindienglish code-mixed data, in: Proceedings of the Second Workshop on Trolling, Aggression and Cyberbullying, 2020, pp. 42–48. [13] B. R. Chakravarthi, N. Jose, S. Suryawanshi, E. Sherly, J. P. McCrae, A sentiment analysis dataset for code-mixed malayalam-english, arXiv preprint arXiv:2006.00210 (2020). [14] B. R. Chakravarthi, R. Priyadharshini, V. Muralidaran, S. Suryawanshi, N. Jose, E. Sherly, J. P. McCrae, Overview of the track on sentiment analysis for dravidian languages in code-mixed text, in: Forum for Information Retrieval Evaluation, 2020, pp. 21–24. [15] A. Das, S. Bandyopadhyay, Subjectivity detection in english and bengali: A crf-based approach, Proceeding of ICON (2009). [16] A. Baruah, K. A. Das, F. A. Barbhuiya, K. Dey, Iiitg-adbu@ hasoc-dravidian-codemixifre2020: Ofensive content detection in code-mixed dravidian text, arXiv preprint arXiv:2107.14336 (2021). [17] B. R. Chakravarthi, R. Priyadharshini, S. Thavareesan, D. Chinnappa, D. Thenmozhi, E. Sherly, J. P. McCrae, A. Hande, R. Ponnusamy, S. Banerjee, C. Vasantharajan, Findings of the Sentiment Analysis of Dravidian Languages in Code-Mixed Text, in: Working Notes of FIRE 2021 - Forum for Information Retrieval Evaluation, CEUR, 2021. [18] R. Priyadharshini, B. R. Chakravarthi, S. Thavareesan, D. Chinnappa, T. Durairaj, E. Sherly, Overview of the dravidiancodemix 2021 shared task on sentiment detection in tamil, malayalam, and kannada, in: Forum for Information Retrieval Evaluation, FIRE 2021, Association for Computing Machinery, 2021. [19] M. Ahmad, S. Aftab, I. Ali, Sentiment analysis of tweets using svm, Int. J. Comput. Appl 177 (2017) 25–29. [20] R. E. Wright, Logistic regression. (1995). [21] T. Tokunaga, I. Makoto, Text categorization based on weighted inverse document frequency, in: Special Interest Groups and Information Process Society of Japan (SIG-IPSJ, Citeseer, 1994. [22] T. P. Trappenberg, Machine learning with sklearn, in: Fundamentals of Machine Learning,
Oxford University Press, 2019, pp. 38–65. [23] B. R. Chakravarthi, R. Priyadharshini, N. Jose, A. Kumar M, T. Mandl, P. K. Kumaresan, R. Ponnusamy, H. R L, J. P. McCrae, E. Sherly, Findings of the shared task on ofensive language identification in Tamil, Malayalam, and Kannada, in: Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, Association for Computational Linguistics, Kyiv, 2021, pp. 133–145. URL: https://aclanthology.org/2021. dravidianlangtech-1.17. [24] A. Hande, R. Priyadharshini, B. R. Chakravarthi, KanCMD: Kannada CodeMixed dataset for sentiment analysis and ofensive language detection, in: Proceedings of the Third Workshop on Computational Modeling of People’s Opinions, Personality, and Emotion’s in Social Media, Association for Computational Linguistics, Barcelona, Spain (Online), 2020, pp. 54–63. URL: https://aclanthology.org/2020.peoples-1.6. [25] S. Suryawanshi, B. R. Chakravarthi, Findings of the shared task on troll meme classification in Tamil, in: Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, Association for Computational Linguistics, Kyiv, 2021, pp. 126–132.
URL: https://aclanthology.org/2021.dravidianlangtech-1.16. [26] B. R. Chakravarthi, N. Jose, S. Suryawanshi, E. Sherly, J. P. McCrae, A sentiment analysis dataset for code-mixed Malayalam-English, in: Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL), European Language Resources association, Marseille, France, 2020, pp. 177–184. URL: https://aclanthology.org/2020.sltu-1. 25. [27] B. R. Chakravarthi, V. Muralidaran, R. Priyadharshini, J. P. McCrae, Corpus creation for sentiment analysis in code-mixed Tamil-English text, in: Proceedings of the 1st Joint Workshop on Spoken Language Technologies for Under-resourced languages (SLTU) and Collaboration and Computing for Under-Resourced Languages (CCURL), European Language Resources association, Marseille, France, 2020, pp. 202–210. URL: https://aclanthology.org/2020.sltu-1.28.