Sarcasm is mainly described as a word that is a synonym for irony. However, it has a more specific meaning: It is commonly used in the context of mocking or conveying contempt. It is considered essential to detect sarcasm in any text on social media because it identifies and conveys the exact meaning of the word and the expected meaning. The Sentiment Analysis System automatically checks the polarity of content, but it doesn't take into account the impact of sarcastic statements. If the system gets it wrong, it won't work as well. So, if it can automatically recognize sarcastic statements from social network data, it can make the Sentiment Analysis System and other NLP-based apps better. In this shared task, we have used the ALBERT transformer model to detect and classify the given text as sarcasm and not sarcasm. Using this model to train and predict the data, we were able to achieve macro F1 scores of 0.48 and 0.34 for the Tamil dataset and Malayalam dataset, respectively.
CEUR ceur-ws.org
Sarcasm is a word derived from the Greek verb ”Sark’azein,” which means to speak bitterly.
These words are often used in a humorous way to mock people [
Amidst all the dificulties and challenges, researchers still try to come up with new ways
to detect sarcasm for various reasons, like to improve communication, because sarcasm often
CEUR
Workshop
Proceedings
leads to miscommunication as the intended meaning of that situation is diferent from the
actual meaning of the word that is used. It also helps with sentiment analysis tasks that help
MNCs and other big companies analyze their product reviews and work accordingly. With the
increasing use of social media, these detection tasks also help in improving cyber security and
wellness [
Sharma et al. [
The task of sarcasm detection was implemented based on two Dravidian languages, namely Tamil and Malayalam. Separate datasets in code-mixed Tamil-English and Malayalam-English were provided by the task organizers for carrying out the task of sarcasm detection. The text in the dataset was represented in Roman and native scripts. Text and label information were provided for each of the instances in the dataset. Text is the actual comment that was posted on social media, and labels define the two main sub-categories in which the comments are grouped, which are sarcastic and non-sarcastic. The training dataset is first fed to the proposed deep learning model. The model uses the data to learn so that it can be used for the purpose of prediction. Later, the model is fed with the validation dataset for further training. This is commonly known as the development dataset. After the training process, the model is fed with instances of the development data, using which it fine-tunes the parameters to increase the accuracy of the results. The last phase of the task involves the use of a test dataset that contains only the text for which the corresponding labels have to be predicted using the trained model. The number of instances under each category of the diferent datasets is shown in Table 1. The training dataset for Tamil and Malayalam languages had 27036 and 12057 samples, respectively. Similarly, the validation dataset had 6759 and 3015 instances in Tamil and Malayalam, respectively, and the test dataset of the Malayalam language contained 3768 comments and the Tamil language contained 8449 comments, for which the labels had to be predicted.
Given a dataset, text classification and prediction are implemented in a sequence of steps. Initially, the training and validation datasets are fed into the system for pre-processing. Various pre-processing techniques, including tokenization, stemming, lemmatization, and the removal of stop words, are implemented, which help in gaining a more accurate result.
The next process is data encoding. The transformer model accepts data in numerical format. Hence, in order to feed the data into the model, the cleaned data is further encoded into numerical data. These encoded data are mapped to the existing words and index values in the model’s vocabulary.
Following this, model selection is done, where the suitable version of the model is chosen to implement the process of classification. The proposed system uses the ALBERT [ 15] model for the process of implementation.
The process of tokenizing data is carried out by the proposed model to satisfy its requirements. Some of the main categories of classifications include segregating the text as classifiers, separators, etc. It is important to ensure that all the tokens generated are of the same length; padding of data needs to be done to rectify the same. Input formatting is also done on the input data. ALBERT models accept the input data to be in the format of segment ID, followed by the attention masks. Segment ID is responsible for diferentiating between the sentence pairs, and attention masks indicate to the model the set of tokens that need attention. Hence, it is necessary that the input data be in this format. Fine-tuning helps the model make predictions based on the encoded input data, which is followed by optimization. The ALBERT model reduces loss and optimizes the output using SOP (sentence order prediction), which reduces loss by avoiding topic prediction. The next significant step in the process is that the model is trained using the dataset, and evaluation is done. Now the model is trained using the development dataset, and the model’s performance is noted. Based on the inference, changes to hyperparameters can be made to achieve better results. As a result of the training process, the model is now made to predict the labels for the instances of the validation dataset, and the comparison of output is done. The architecture of the proposed model is represented by Figure 1.
Finally, the test data which is the new unseen data is fed into the model and labels are generated for the data. Compared to other BERT models we have used the ALBERT model for classification purposes as it supports Indian languages and classifies text in an eficient way by parameter sharing which reduces overfitting and computation is done faster. This model is also highly scalable in nature which makes it versatile. 4.1. ALBERT ALBERT [15] stands for ”A Lite BERT” as it is extracted from the BERT model. BERT (Bidirectional Encoder Representations from Transformers) is a transformer model that uses transformer encoders to process the given input data. Both BERT and ALBERT models use the same backbone architecture represented by Figure 2.The advantages of using ALBERT over BERT are that its computational speed is fast, and it is also stated that ALBERT performs better even with a smaller number of parameters, unlike BERT. The number of parameters is reduced by the parameter sharing method and the factorization of the embedding matrix. Using this method, the embeddings generated are split into two matrices. Input-level embeddings will have all the embeddings that will process the context-independent learning. Similarly, high-level embeddings are responsible for context-dependent learning. ALBERT is a supervised learning model, meaning it learns from the given input dataset and trains the model based on its learning. Albert uses masked language models to train the data. This model makes use of the self-supervised sentence order prediction loss to find out the inter-sentence relations in the given input data.
ALBERT is a pre-trained model, and hence performing operations is made easy using the TensorFlow hub.
Table 2 shows the results of the Sarcasm detection task that was carried out. Using the proposed model, we were able to predict the labels for the comments given in the dataset. It yielded an accuracy of 0.81 and 0.79 for the Tamil and Malayalam datasets, respectively. It could be seen that out of 8452 comments, 1883 comments are sarcastic and 6567 comments are non-sarcastic in the Tamil dataset. Similarly, in the Malayalam dataset, out of 3768 comments, 69 are sarcastic and 3699 are non-sarcastic. Macro-F1 scores of 0.48 and 0.34 were also achieved in the Tamil and Malayalam datasets, respectively. The classification report obtained for Tamil and Malayalam are represented by heatmaps in the Figure 3 and Figure 4
While comparing the predicted labels obtained using the proposed model and the actual labels for each instance of the dataset, it was found that there are both false positive and false negative values. This can be further witnessed with the F1-score that is obtained during the process.
The reasons for the error in the predictions could be due to the absence of any sarcastic word; thus, it is classified as ”non-sarcastic” instead of the appropriate label of ”sarcastic”. Another reason could be that few texts do not have words in them but rather just symbols; hence, it is dificult to predict the correct label. Hence, such texts are classified as non-sarcastic instead of the actual label ”sarcastic”. All the example texts demonstrate sarcasm and play a significant role in the classification process. Sample text instances that are misclassified are represented in tables 3 and 4
The way people communicate online is getting more and more complicated. So traditional methods like feature-based or machine-learning-based methods won’t work if you’re trying to detect sarcasm. It’s important to diferentiate between sarcastic and non-sarcastic text when it comes to online content. Trying to detect sarcasm by looking at things like language, sentiment, and syntax can give people the wrong idea. Context and semantic information are key when it comes to spotting sarcasm. We want to make our work better in the future by using a bigger dataset for training. Plus, emojis and emoticons are really important for showing what a comment means on social media, so we’ll think about adding them to the text. [13] S. K. Lora, G. Shahariar, T. Nazmin, N. N. Rahman, R. Rahman, M. Bhuiyan, et al., Ben-sarc: A corpus for sarcasm detection from bengali social media comments and its baseline evaluation (2022). [14] J. Mahibha, S. Kayalvizhi, D. Thenmozhi, Ofensive language identification using machine learning and deep learning techniques (2021). [15] Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, R. Soricut, Albert: A lite bert for self-supervised learning of language representations, arXiv preprint arXiv:1909.11942 (2019).