Sarcasm recognition is one of the most dificult problems in natural language processing (NLP), especially for low-resource languages like Tamil and Malayalam that have unique linguistic and cultural traits. A hybrid model that includes several neural network architectures and a Bidirectional Long Short-Term Memory (Bi-LSTM) model are the two deep learning techniques for sarcasm identification in various languages that are studied in this work. It takes skill to record context in both forward and backward directions, which the Bi-LSTM model aptly demonstrates, in order to comprehend the intricate linguistic structure of sarcastic statements. Using long-range dependencies in addition to local feature extraction, the hybrid model combines multiple architectures to improve sarcasm detection. We use comprehensive preprocessing techniques using Malayalam and Tamil sarcasm datasets, including tokenization, padding, and label encoding. Our hybrid model outperformed the Bi-LSTM in accuracy and F1-scores, ranking 5th in the Tamil dataset with an MF1 score of 0.70 and 7th in the Malayalam dataset with an MF1 score of 0.67. These findings demonstrate how dificult it is to use irony in these languages and how important hybrid architectures are for overcoming dificulties in low-resource languages. This paper shows the efectiveness of deep learning models in sarcasm detection and lays the groundwork for future sentiment analysis research.
Sarcasm is a sophisticated and subtle style of speech in which a sentence’s literal meaning frequently difers from its intended meaning. Sarcasm is commonly employed in everyday interactions, especially on social media, where people use it to convey irony, humor, or criticism. Natural language processing (NLP) systems find it dificult to identify sarcasm in written text because it necessitates a grasp of the words themselves as well as their context, tone, and very subtle cultural undertones.
When it comes to tackling NLP tasks, deep learning models have demonstrated a lot of promise. Long Short-Term Memory (LSTM) networks, for example, have proven to be useful in capturing contextual information in sequential data. Bidirectional LSTMs, or Bi-LSTMs, have drawn particular attention because of its capacity to recognize both past and future context in a sentence. This property makes Bi-LSTMs perfect for identifying complicated expressions like sarcasm, which frequently require comprehending context from many perspectives. Sarcasmic expressions, however, can be buried in linguistic or cultural quirks that are dificult for a single model architecture to represent, which makes sarcasm identification in low-resource languages more dificult.
This research investigates two deep learning-based methods for sarcasm identification in Tamil and Malayalam, a hybrid model and a bidirectional LSTM model, in order to overcome these issues. The BiLSTM model makes use of its capacity to sequentially record context from both sides, whereas the hybrid model incorporates several neural network designs to improve performance through the simultaneous
CEUR
ceur-ws.org
capture of textual information that are local and global[
The paper by Farhan [
In a shared challenge on code-mixed Dravidian languages, [
The paper by [
The study by Gulecha [
This paper, Thandil [13] develops an end-to-end multi-dialect Malayalam speech recognition system with machine learning, LSTM-RNN, and deep-CNN. Accent diferences in speech are addressed by the hybrid technique, which improves recognition performance[14]. In order to identify abusive language in Tamil, Malayalam, and Kannada, this study used deep learning models like Bi-LSTM and hybrid networks with convolutional layers and Bidirectional RNNs. The training and prediction performance of these models was enhanced by[15].
The challenge lies in identifying sarcasm in Tamil and Malayalam, given their distinct linguistic characteristics. To increase F1-scores and the accuracy of sarcasm detection, the system makes use of a hybrid model that combines several neural networks and a bidirectional LSTM.
The dataset used for this study came from the Codalab website, which has YouTube video comments with code-mixed Tamil-English and Malayalam-English sentences. This dataset helps develop a machine learning model for Malayalam and Tamil sarcasm detection [16].
Tokenization is used in text preprocessing to divide text into tokens, padding is used to guarantee a consistent sequence length, and label encoding is used to translate categorical labels into numerical values. Once tokens are converted into dense vectors appropriate for model input, word embeddings are applied.
The methodology uses a hybrid model and a bidirectional LSTM (Bi-LSTM) model to identify sarcasm in Tamil and Malayalam text. These methods are intended to capture contextual elements in text data, both local and global, in order to handle the special dificulties associated with sarcasm identification.
Embedding Layer: Dense word embeddings are created from raw text in this first layer. In a continuous vector space, every word is represented as a vector that captures its semantic meaning and relationships with other words. The model can comprehend and analyze text in a numerical format thanks to this representation.
Bidirectional LSTM Layer: The Bidirectional Long Short-Term Memory (Bi-LSTM) network is the central component of the Bi-LSTM concept. A Bi-LSTM simultaneously processes sequences in both directions, in contrast to a normal LSTM, which processes text in one direction (either forward or backward). Understanding complex expressions like sarcasm requires the model to be able to capture context from both the past and the future inside a sequence, which is made possible by this bidirectional processing. • Forward LSTM: Completely processes the input sequence. • Backward LSTM: Processes the input sequence from end to beginning.
By combining the results from both sides, the model is able to comprehend context more thoroughly.
Dropout Layer: Applied after the LSTM layer, this layer randomly eliminates a portion of the neurons during training in order to minimize overfitting. By doing this, the model is kept from becoming overly dependent on particular neurons and is able to more efectively generalize to new data.
Dense Layers: The last classification is carried out using dense layers, which come after the Bi-LSTM and dropout layers. Using activation functions like Softmax or Sigmoid, these layers transfer the information collected by the LSTM layers to output classes (such as sarcastic or non-sarcastic).
Embedding Layer: Text is first transformed into dense vectors that represent the semantic meaning of words by the model’s embedding layer. As a result, the network can operate using numerical representations of the word relationships.
Components of Multi-Head Attention: The hybrid model includes Multi-Head Attention, which enables the model to focus on several textual elements at once. This facilitates the identification of subtle sarcastic patterns and long-range dependencies within the text. The attention mechanism makes sure that important terms or phrases that indicate sarcasm are highlighted in the text so that it can be understood more quickly.
LSTM Components: The model incorporates bidirectional LSTM layers to capture the sequential lfow and relationships between words. These layers ensure that the model comprehends the context around the text’s sarcastic expressions by processing information both forward and backward.
Transformer Block: Enhancing the attention mechanism even further, the hybrid model allows for more efective learning by incorporating a transformer block with Add & Normalize layers. Through improved feature extraction from the input sequence, this stabilizes the network.
Feed Forward Network (FFN): To extract higher-level patterns, an attention mechanism and LSTM are followed by a tiny, fully connected network. at order to classify sarcasm at the end, this layer assists the model in integrating the features acquired from the attention and LSTM components.
Layer of Output: The last layer employs a sigmoid activation function to produce the likelihood that the text in question includes irony. In order to assist the model recognize sarcasm accurately, it uses a binary classification.
5.1. Bi-LSTM The sarcasm detection datasets in Tamil and Malayalam were used to assess the Bi-LSTM model. The model performed somewhat better, with an accuracy of 0.80 for the Malayalam dataset, than it did for the Tamil dataset, where it had an accuracy of 0.76. The Bi-LSTM model showed a moderate degree of success in identifying sarcasm in both languages due to its capacity to capture both forward and backward dependencies. Though it worked well, it could not fully capture the complex patterns of sarcasm.
The hybrid model scored better in both datasets than the Bi-LSTM model, combining elements of Transformer and LSTM. Outperforming the Bi-LSTM with an accuracy of 0.78 on the Tamil dataset, the hybrid model performed better. Even better, the hybrid model achieved an accuracy of 0.83 on the Malayalam dataset. Sarcasm might be detected more accurately thanks to this model’s capacity to record both local characteristics using Transformer and long-range dependencies using LSTM. In addition, with an F1 score of 0.70 on the Tamil dataset and 0.67 on the Malayalam dataset, the hybrid model came in fith place.
The hybrid model demonstrated its efectiveness by ofering a reliable method for recognizing sarcastic phrases in Malayalam and Tamil. Hybrid Transformer-LSTM Network and Long Short-Term Memory (LSTM) networks were integrated, and this greatly improved performance. The model’s accuracy levels were noteworthy. Because of its dual architecture, the model is an excellent candidate for jobs needing a sophisticated grasp of sarcasm. It can successfully capture both local aspects and long-range contextual relationships. Furthermore, the Bi-LSTM model showed its ability to identify languages and extract features, although it had trouble understanding the nuances of sarcasm in languages with little resources. Overall, the hybrid model outperforms the other models, demonstrating both its versatility for diferent language tasks and its promise for more accurate sarcasm detection.
During the preparation of this work, the author(s) used ChatGPT in order to: drafting content, grammar and spelling check, etc. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content. for tamil language, in: International Conference on Speech and Language Technologies for Low-resource Languages, Springer, 2023, pp. 225–235. [12] S. S, P. K. V, Hybrid deep learning approach for sentiment classification of malayalam tweets, International Journal of Advanced Computer Science and Applications (2022). doi:10.14569/ ijacsa.2022.01304103. [13] R. K. Thandil, K. Mohamed Basheer, V. Muneer, End-to-end multi-dialect malayalam speech recognition using deep-cnn, lstm-rnn, and machine learning approaches, in: International Conference on Computational Intelligence and Data Engineering, Springer, 2022, pp. 37–49. [14] B. R. Chakravarthi, A. Hande, R. Ponnusamy, P. K. Kumaresan, R. Priyadharshini, How can we detect homophobia and transphobia? experiments in a multilingual code-mixed setting for social media governance, International Journal of Information Management Data Insights 2 (2022) 100119. [15] S. K, B. Premjith, S. Kp, Amrita_cen_nlp@dravidianlangtech-eacl2021: Deep learning-based ofensive language identification in malayalam, tamil, and kannada, 2021. [16] B. R. Chakravarthi, Hope speech detection in youtube comments, Social Network Analysis and Mining 12 (2022) 75.