Ofensive language identification in countries like India poses several challenges due to the usage of code-mixed and romanized variants of multiple languages by the users in their posts on social media. The challenge of ofensive language identification on social media for Dravidian languages is harder, considering the low resources available for the same. In this paper, we employed two multilingual transformer-based language models-multilingual BERT (mBERT) and XLM-RoBERTa-to perform ofensive language identification in Dravidian code-mixed languages. The languages in the task are romanized variants of four Dravidian languages: Malayalam, Tamil, Tulu, and Kannada. The proposed methodology makes use of Transfer Learning (TL) based Multilingual Bidirectional Encoder Representations from Transformers (mBERT) model and XLM-RoBERTa. These models achieved macro F1 scores of 0.475, 0.774, 0.453, and 0.819, securing 1st, 1st, 1st, and 2nd ranks for Kannada, Tamil, Tulu, and Malayalam languages, respectively. The proposed models yielded good results and are promising for efective ofensive speech identification in low resource settings.
Ofensive speech is defined as speech that causes a person to feel upset, resentful, annoyed, or insulted.
In recent years, social media such as Twitter, Facebook and Reddit have been increasingly used for
the propagation of ofensive speech and the organization of hate and ofense-based activities [
Until a few years ago, hate and ofensive speech were identified manually which is now an impossible task due to the enormous amounts of data being generated daily on social media platforms. The need for scalable automated methods for detecting hate speech has attracted significant research in the domains of natural language processing and Machine Learning. Researchers develop and experiment with a variety of techniques and tools such as bag of words models, N-grams, dictionary-based approaches, and word sense disambiguation techniques. Recent developments in multilingual text classification are led by Transformer architectures like mBERT and and XLM-RoBERTa.
An additional advantage of these architectures, particularly XLM-RoBERTa, is that they give good results even with lower-resource languages, and this particular aspect is beneficial to Indian languages, which do not have properly established datasets. In our work, we focused on using these architectures in multiple ways. However, there is a limitation in directly applying these models to romanized or code-mixed text. The transformer models are trained on languages in their native script, not in the romanized script in which users prefer to write online.
This shared task presents a corpus for ofensive language identification of code-mixed text in Dravidian languages (Tamil-English, Malayalam-English, Kannada-English, and Tulu-English). This task is further complicated in low-resource languages where limited annotated datasets exist for ofensive speech detection.
The paper is organized as follows: Section 2 explores related work, followed by an overview of the datasets in Section 3. Section 4 details the methodology, and Section 5 analyzes the results. The paper wraps up with conclusions in Section 6.
Researchers used a wide variety of techniques for the identification of ofensive languages. Few of the research studies are described below.
Mishra et al.[
In [
The work [
Limited research has been conducted on ofensive language identification for Dravidian languages to date. This work addresses the existing research gap by developing methods for identifying ofensive speech in Dravidian languages. The proposed systems are designed to be adaptable, with potential extensions to other Indian and international languages.
This shared task presents a gold-standard dataset for ofensive language detection in Tamil, Malayalam, Kannada, and Tulu, enabling researchers to develop robust classification models. The more details of the shared task are displayed in the task website 2. The dataset consists of social media comments and posts that are categorized into four classes: 1http://huggingface.co/google-bert/bert-base-multilingual-cased 2https://dravidian-codemix.github.io/2025/ • Not Ofensive (NO): Content without any ofensive elements. • Ofensive Untargeted (OU): Ofensive content that is not directed at a specific individual or entity. • Ofensive Targeted (OT): Direct attacks on an individual or group, including hate speech targeting a community, ethnicity, caste, or gender. • Not Tamil/Not Malayalam/Not Kannada/Not Tulu (NT): Content that does not contain the Tamil,
Malayalam, Kannada, or Tulu languages.
The primary goal of this shared task is to build and evaluate systems that can automatically classify social media text into these four categories. Participants will be provided with training, development, and test datasets to develop their models. Given the real-world class imbalance in ofensive content, models must be designed to handle the skewed distribution of data efectively.
In this study, we employed two multilingual transformer-based language models–multilingual BERT (mBERT) and XLM-RoBERTa–to perform multilingual text classification tasks. These models are selected due to their proven eficacy in handling diverse languages and contextual embeddings. To address potential class imbalance in the dataset, we experimented with class weighting techniques during training, which adjust the loss function to penalize misclassifications of underrepresented classes more heavily. Specifically, class weights were computed inversely proportional to the class frequencies in the training data.
• mBERT with class weights : This setup utilized the base mBERT model (bert-base-multilingualcased) fine-tuned with class weights applied to the cross-entropy loss. This configuration yielded the best overall performance, achieving superior accuracy, F1-score, and generalization across languages, likely due to mBERT’s balanced multilingual pre-training combined with efective handling of class imbalance. • XLM-RoBERTa with class weights: We fine-tuned the base XLM-RoBERTa model (xlm-robertabase) incorporating the same class weighting strategy. While it demonstrated strong results, particularly in cross-lingual transfer, it underperformed compared to the mBERT variant with weights, possibly owing to diferences in pre-training objectives (e.g., XLM-RoBERTa’s focus on masked language modeling across more languages but with less emphasis on next-sentence prediction). • mBERT without class weights: This served as a baseline, fine-tuning mBERT without any class weighting, relying solely on standard cross-entropy loss. This approach resulted in noticeably diferent predictions, with a bias toward majority classes and reduced performance on minority classes, highlighting the importance of class weights in mitigating imbalance issues.
These models are fine-tuned for ofensive language detection. This involves training the model on
taskspecific labeled data. During both pretraining and fine-tuning, the model utilizes attention mechanisms
to process the input text. It considers the context of each word by attending to its surrounding words,
capturing long-range dependencies efectively. Thus, mBERTmodel is designed to provide a powerful
and flexible framework for multilingual NLP tasks such as SA[
Statistics of the dataset provided by the organizers of the shared task for the Ofensive Language Identification in Dravidian Languages for Tasks are shown in Tables 1. bert-base-multilingual-cased- a mBERT model from the huggingface repository is used to extract the feature vectors. After loading the pre-trained mBERT model with its default parameter values, the model is frozen to prevent further updates to its weights.
The models are evaluated based on weighted F1 scores by incorporating class weights. Among the proposed models, mbert model with weights obtained better results. The proposed models obtained macro F1 scores of 0.475, 0.774, 0.453, and 0.819, securing 1st, 1st, 1st, and 2nd ranks for Kannada, Tamil, Tulu, and Malayalam languages, respectively.
From the Table 2, it is clear that the submitted predictions has exhibited better weighted F1 scores comparing to other teams by securing top ranks. The proposed methodology has still exhibited low macro F1 score for Kannada and Tamil text. The performance of the classifier across all four languages is illustrated using confusion matrices in Figure 1. These matrices highlight not only the correctly classified samples along the diagonal, but also the types of misclassifications that occur between classes. It can be observed that most of the wrong classifications are due to lack of context. Table 3 shows the sample text from Tulu and Kannada labeled Test sets along with their the actual and predicted labels. The proposed model achieved lower macro F1 scores for Kannada and Tulu compared to the other two languages evaluated. This reduced performance can be attributed to the significantly smaller dataset sizes available for Kannada and Tulu as shown in Table 1. The limited training data likely hindered the model’s ability to actually capture the linguistic nuances and code-mixed patterns prevalent in these low-resource languages.
(a) Kannada
(b) Malayalam (c) Tamil (d) Tulu
This paper describes the models submitted to the shared task ”Ofensive Language Identification in
Dravidian Languages” at FIRE 2025 [
During the preparation of this work, the author(s) used Chat GPT-4 in order to:Grammar and spelling check. Using this tool, the author(s) reviewed and edited the content as needed and take full responsibility for the publication’s control