Sentiment analysis deals with analysing the given text and classifying whether the text is a positive, negative or neutral one. The sentiment analysis forms the base for applications where the public views could be known. This paper shows how a multi label classification of the given text could be implemented by considering the sentiment associated with the text. The models that are applied for monolingual sentiment analysis may not provide good results when it is extended for code mixed data. When a Cross Lingual model was applied to the training data-set provided by Dravidian Code-mix FIRE 2021 for the Task A which uses Tamil - English code mixed data, it was able to classify the test data-set with an average F1 score of 0.514. .
India is a land of large bilingual communities. Most part of the Indian community are well
versed in the language English [
Sentiment analysis is a research area under Natural language Processing where the sentiment
associated with a text has to be identified [
When a corpus is created for a specific purpose like Sentiment analysis, the data in the
corpus may not be uniformly distributed. It may have a particular class of data much more than
nEvelop-O
(D. Thenmozhi)
other classes or a particular class of data may be much smaller than the other classes[
During this pandemic period, exposure to social media has seen a wide increase. Due to the use of online resources and online media specifying their views and posting comments in Social media sites have become very common among people. Indians prefer to express themselves using code mixed languages. These are dificult to analyse as they may not be grammatically formed with variations in spelling and use of abbreviations and also it has limited resources [8]. Deep learning models could be efectively utilized for performing the multi label classification of sentiment analysis over the code mixed data. Cross-lingual pretrained transformer models are expected to provide better performance for such task when it is associated with low-resourced languages [9].
In a code mixed corpus, usage of both code-mixed and non-code-mixed data are common
which had been identified by creation of a lexicon dictionary [
Sentiment analysis of Kannada-English code mixed corpus which had been created by crawling Facebook comments and the performance of various machine learning and deep learning techniques over the corpus using a distributed representation had been demonstrated by [10]. Sentiment analysis of a Hindi data-set had been implemented using the concept of cross-lingual contextual word embeddings and zero-shot transfer learning to project the predictions from resource-rich English to resource-poor Hindi language [11]. Classification of code mixed Hindi text based on sentiment had been implemented with the use of TF-IDF feature vectors of character n-grams where n ranged from 2 to 6 with an ensembled voting classifier and linear SVM classifier[ 12].
Polarity of Dravidian code-mixed comments had been identified using a sub-word level model and a word embedding based model which in turn had made use of Long Short Term Memory (LSTM) network and a machine learning based architecture which had used Inverse document frequency (TF-IDF) vectorization along with a Logistic Regression model [13] for the classification task. The Decision Tree Algorithm had been used to computationally identify and categorize the opinions expressed as text in Kannada language [14].
To analyse code mixed data, bilingual embedding techniques has to be replaced with multilingual word embedding schemes [15] to achieve improvement in performance in any application associated with code-mixed data. For implementing sentiment analysis of code-mixed data, using diferent kinds of multilingual and cross-lingual embeddings, knowledge can be eficiently transferred from monolingual text to code-mixed text [16]. Variations of code mixed words had been captured by a cluster based preprocessing approach and then the sentences of code-mixed and standard languages had been mapped to a common sentiment space for performing the sentiment analysis by [17].
The data-set provided for the shared task Dravidian Code-Mix FIRE 2021 [
The table 1 shows the number of instances under each category both in the training and the validation data set. Considering the distribution of data in the data set it showed an imbalanced nature of the real world scenario with 56% of the data falling under Not-Known category in both training and validation sets. The instances of the test data-set has to be placed under any one of the above categories which was done by predictions based on the proposed model. The average F1 score of this predicted class of the test data was used by Dravidian Code-Mix FIRE 2021 to evaluate the proposed model and based on the predicted values, average F1 score was computed and the models were ranked.
The Fig:1 shows the distribution of the data in both the training and validation data-set provided by Dravidian Code-mix Fire 2021 for the Task1. It was evident that the data-set is an imbalanced one with more instances under the category Not- Known.
As separate data-sets were provided for training and validation purposes, they were used for training and validating the proposed model. A separate test data-set was provided for which the sentiments were predicted using the trained model which were used for computing the F1 score of the model.
The shared task Dravidian Code-Mix FIRE 2021 was a multi-label classification task for sentiment analysis of Dravidian Code-mixed data based on the provided gold standard corpus[? ]. We as a team participated in the task associated with the sentiment analysis of Tamil-English code mixed text which had comments from YouTube. We have implemented the classification using a Cross Lingual pre-trained model.
The given data-set was imbalanced in nature, as a result of which the model was not able to turn up with good results. The data from the training data-set was down-sampled to enforce a balanced nature to the data-set. Using the Cross Lingual model substantial improvement in performance of the model is expected over both low-resource languages and high-resource languages. The Cross Lingual Model is trained with a Translation Language Modeling which helps the model to learn similar representations for diferent languages 1. The vocabulary supported by the model was the default value which is 30145, and it uses 2048 encoder and pooling layers and 12 hidden layers with a dropout value of 0.1. The model supports language embedding with two languages supported by the model as the considered data set has text in code-mixed Tamil and English. The model was trained using the training data set and validated 1https://huggingface.co/transformers/model_doc/xlmroberta.html using the validation data set provided for the shared task by Dravidian Code-Mix FIRE 2021. The evaluation of the model was based on a separate test data set.
The performance of Dravidian Code-mix FIRE 2021 was evaluated based on the performance of the proposed system which in turn was measured in terms of weighted averaged precision, weighted averaged recall and weighted averaged F-Score across all the classes. Our model was able to provide an average F1 score of 0.514. Table 2 shows the value of various performance measures for sentiment predictions done for the text in the test data-set using the proposed model.
The performance measures of the model shows that there is much more scope to improve the accuracy of the sentiment analysis of code mixed Tamil-English text. Imbalance nature of the data-set could be considered as one possible reason for this output which could be balanced by using up-sampling mechanisms instead of down-sampling the data. The down-sampled data could have made the model to miss out few key features which would have led to missclassification of the text from the test data-set. Pre-processing of the data is equally important as up-sampling of the system as the posts retrieved from YouTube would have symbols and abbreviations.
From the result it could be found that the performance achieved by our model on the new golden corpus provided by Dravidian Code-mix FIRE 2021 for code mixed Tamil-English text could be improved. As numerous multilingual based transformer models and approaches are available, more scope is there for research to be carried out in this field to identify a better model for sentiment analysis of code mixed text. [8] S. R. Shah, A. Kaushik, Sentiment analysis on indian indigenous languages: a review on multilingual opinion mining, arXiv preprint arXiv:1911.12848 (2019). [9] G. Lample, A. Conneau, Cross-lingual language model pretraining, arXiv preprint arXiv:1901.07291 (2019). [10] K. Shalini, H. B. Ganesh, M. A. Kumar, K. P. Soman, Sentiment analysis for code-mixed indian social media text with distributed representation, in: 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI), 2018, pp. 1126–1131. doi:1 0 . 1 1 0 9 / I C A C C I . 2 0 1 8 . 8 5 5 4 8 3 5 . [11] A. Kumar, V. H. C. Albuquerque, Sentiment analysis using xlm-r transformer and zeroshot transfer learning on resource-poor indian language, Transactions on Asian and Low-Resource Language Information Processing 20 (2021) 1–13. [12] P. Mishra, P. Danda, P. Dhakras, Code-mixed sentiment analysis using machine learning and neural network approaches, arXiv preprint arXiv:1808.03299 (2018). [13] A. V. Mandalam, Y. Sharma, Sentiment analysis of Dravidian code mixed data, in: Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages, Association for Computational Linguistics, Kyiv, 2021, pp. 46–54. URL: https://aclanthology.org/2021.dravidianlangtech-1.6. [14] P. Ranjitha, K. Bhanu, Improved sentiment analysis for dravidian language-kannada using dicision tree algorithm with eficient data dictionary, in: IOP Conference Series: Materials Science and Engineering, volume 1123, IOP Publishing, 2021, p. 012039. [15] A. Pratapa, M. Choudhury, S. Sitaram, Word embeddings for code-mixed language processing, in: Proceedings of the 2018 conference on empirical methods in natural language processing, 2018, pp. 3067–3072. [16] S. Yadav, T. Chakraborty, Unsupervised sentiment analysis for code-mixed data, arXiv preprint arXiv:2001.11384 (2020). [17] N. Choudhary, R. Singh, I. Bindlish, M. Shrivastava, Sentiment analysis of code-mixed languages leveraging resource rich languages, arXiv preprint arXiv:1804.00806 (2018).