The dramatic surge in the use of social media platforms for sharing information and opinions has, unfortunately, also fueled a sharp rise in online abuse. One of the simplest yet most insidious tools for such abuse is the meme: a visual artifact that typically fuses an image with a short, often provocative, text overlay. While memes can be humorous or satirical, they are increasingly being weaponized to target individuals or communities through hateful or derogatory content. This poses a serious threat to online safety and digital well-being. Detecting and curbing such abusive memes is therefore an urgent necessity. However, the task becomes significantly more challenging in low-resource Indian languages such as Bangla, Hindi, Gujarati, Bodo etc., where annotated benchmark datasets are scarce or entirely absent. Without such datasets, the development and evaluation of robust AI models remain severely constrained. In this work, we aim to bridge this crucial gap by constructing a multilingual abusive meme dataset-covering Bangla, Hindi, Gujarati, and Bodo, thereby enabling research and innovation in abusive meme detection across low-resource Indian languages. Each dataset is annotated with five classification labels: sentiment, sarcasm, vulgarity, abuse, and target. A total of 20 unique teams participated, submitting over 306 runs across the four languages. The performance of the best systems was evaluated using the Macro F1 score, with the top-performing models achieving scores of 0.6275 (Bangla), 0.6570 (Hindi), 0.6750 (Gujarati), and 0.6312 (Bodo). This article briefly summarizes the tasks, data development, results and approaches.
The concept of memes was first proposed by Richard Dawkins in 1976, describing them as cultural
elements that spread from person to person much like genes, quickly sharing ideas and influencing
group thinking [
CEUR Workshop
ISSN1613-0073
as a whole. In today’s world, “Internet memes” or “image memes” are widely seen on social media
platforms. A meme is typically a visual idea that combines an image with a short caption written over
it, making the text an essential part of the picture [
The HASOC (Hate Speech and Ofensive Content Identification) track, which has provided a platform
for hate speech detection since 2019 at FIRE (Forum for Information Retrieval Evaluation) [
Research on multimodal abusive or hateful memes has grown steadily, supported by the development
of several datasets over the last few years. One of the earliest initiatives was by Sabat et al. [
A major milestone in this space is the Hateful Memes Challenge dataset created by Facebook AI [
The release of this dataset catalyzed a series of multimodal shared tasks. SemEval-2022 Task 5
(MAMI) [
In contrast to English, multimodal meme datasets for Indian languages are limited and often not publicly
available. Karim et al. [
Emofmeme [
TamilMemes [31] is a Tamil-language dataset containing 2,969 memes created for troll meme detection. In this dataset, trolling is annotated as a dedicated dimension using a binary labeling scheme (yes/no).
While most existing eforts focus on English, abusive meme detection in Indian languages remains largely underexplored, with the exception of Bengali and Hindi. To bridge this gap, our work presents a multimodal abusive-meme dataset for Bangla, Hindi, Gujarati, and Bodo, designed to closely reflect memes typically shared on real social media platforms. The scarcity of datasets, limited transparency, and lack of resources that capture real-world meme distributions strongly motivate the development of high-quality datasets for Indian languages, which forms the core motivation of this study.
In HASOC meme 2025, the task is with four languages proposed in the research area of hate speech detection. These tasks ofered all four languages: Bangla, Hindi, Gujarati, and Bodo. Figure 1 shows the Screenshot of HASOC meme Website 2.
This task involves analyzing multimodal data (image and text) to detect abuse, identify targeted communities, assess vulgarity and sarcasm, and assign sentiment labels. So, the task will be in five parts. (1) Sentiment detection: positive, negative, and neutral, (2) Sarcasm Detection: sarcastic or not, (3) Vulgarity Detection: vulgar or not, (4) Abuse Detection: abusive or not, and (5) Target Community Identification: Gender, Religion, Individual, Political, National Origin, Social Sub-groups, Others, and none.
In this section, dataset collection, annotation, and analysis have been discussed for HASOC-meme.
Our primary aim in constructing this multilingual, multimodal dataset is to ensure its diversity, so we
intentionally selected some Gender, Religion, Political, communal ™Facebook meme pages, ™Instagram,
™Google Image, ™Bing, and ™YouTube channels etc. Unlike the Hateful Memes Challenge [
For the annotation process, we engaged fourteen undergraduate students (aged 20–25 years), consisting of nine males and five females, to identify whether each meme was abusive or non-abusive. These 2https://hasocfire.github.io/hasoc/2025/call_for_participation.html (Access on 05.12.2025)
Determine whether the meme conveys sarcasm or irony Identify vulgar or obscene language/imagery Determine whether the meme is abusive
Assign a sentiment la- Positive: Supportive, humorous, or appreciative tone bel to the meme Neutral: Neither positive nor negative
Negative: Hostility, mockery, or criticism Sarcastic: The meme conveys sarcasm or irony Non-Sarcastic: The meme does not convey sarcasm or irony Vulgar: Explicit or ofensive words/gestures Not Vulgar: No explicit or ofensive content Abusive: Harmful, ofensive, or derogatory content targeting individuals or groups
Non-Abusive: No harmful or derogatory content Identify the commu- Gender: Male, female, non-binary, transgender nity targeted by the Religion: Religious beliefs, deities, practices meme, if any Individual: Targeting a specific person.
Political: Political ideologies, parties, people National Origin: Country or ethnicity-based groups Social Sub-groups: Socio-economic, cultural, occupational groups Others: Any target not in above categories
None: No specific target
annotators worked across all four languages, like Bangla, Hindi, Gujarati, and Bodo, and were supported
with OCR-extracted text. All students were native speakers of the respective languages and were
compensated fairly according to local standards. The entire annotation workflow was overseen by a
post-doctoral researcher, Ph.D. researchers, and an industry-academic researcher, each with several
years of experience in analyzing harmful social media content. We adopt the same schemes as the
authors [
Figure 2 shows the train and test split of the Bangla, Hindi, Gujarati, and Bodo datasets. We summarize the key statistics of our multilingual meme traning datasets in Table 2. Across all four languages, noticeable label imbalance is present in every annotation category. In the Bangla dataset, negative sentiment (1,476 samples) is substantially higher than positive (906) and neutral (311) classes. Hindi exhibits a similar pattern, where negative sentiment dominates the other two classes. In the Gujarati, more positive and neutral labels are present than negative. The Bodo dataset is particularly imbalanced, containing no neutral samples and showing a clear skew toward the positive class (227 positive vs. 151 negative).
A strong imbalance is also observed in sarcasm labels. Bangla contains a significantly higher proportion of sarcastic memes (2,081) compared to non-sarcastic ones (612). Hindi (770 vs. 371) and Gujarati (670 vs. 219) show similar skewed distributions. In the Bodo dataset, this imbalance becomes even more pronounced, with 339 sarcastic and only 39 non-sarcastic samples.
Vulgarity labels show imbalance in the opposite direction for most languages. All four datasets have considerably more non-vulgar than vulgar samples. For instance, Bangla (2,226 non-vulgar vs. 467 vulgar), Hindi (764 vs. 377), Gujarati (592 vs. 297), and Bodo (271 vs. 107), indicating that vulgar memes represent a minority class.
(a) (b)
Abuse labels also display asymmetry. While Bangla and Hindi contain more non-abusive than abusive samples, the gap is large (Bangla: 1,954 vs. 739; Hindi: 834 vs. 307). Gujarati follows the same trend, whereas Bodo has a relatively lower but still imbalanced distribution (381 non-abusive vs. 77 abusive).
Overall, each of the four languages demonstrates clear class imbalance across sentiment, sarcasm, vulgarity, and abuse labels, which highlights the need for careful modeling strategies and balanced evaluation. Figure 3 provides a graphical visualization of these distributions.
The macro F1-score computes the F1-score independently for each label: sentiment, abuse, vulgarity, sarcasm, then averages them. This metric places greater emphasis on minority classes, penalizing systems more heavily when performance is weak on underrepresented labels. The choice of an F1 variant depends on the task objectives and class distribution; however, due to the significant class imbalance typically present in hate speech detection, the macro F1-score is the most appropriate evaluation measure. For participant’s system run submissions and evaluation in our task, we rely on the Kaggle platform. Figure 4 displays a screenshot of the Kaggle leaderboard used for run submissions. Separate Kaggle competition pages are provided for Bangla 3, Hindi 4, Gujarati 5 and Bodo 6 to enable participants to submit their experimental results.
Overall, 45 participants register for the task HASOC meme. In the Bangla task, 17 teams made 69 submissions, while 18 teams submitted 82 runs in the Hindi task, for Gujarati 15 teams made 68 submissions, and for the Bodo task, 15 teams submitted a total of 87 runs.
The performance of the best classification algorithms for Bangla, Hindi, Gujarati, and Bodo are Macro F1 measures of 0.6275, 0.6570, 0.6750, and 0.6312, respectively. The results for Bangla, Hindi, and Gujarati datasets are shown in Table 3, Table 4, Table 5, and Table 6, respectively. 3https://www.kaggle.com/competitions/hasoc-2025-meme-bangla (Access on 05.12.2025) 4https://www.kaggle.com/competitions/hasoc-2025-meme-hindi (Access on 05.12.2025) 5https://www.kaggle.com/competitions/hasoc-2025-meme-gujarati (Access on 05.12.2025) 6https://www.kaggle.com/competitions/hasoc-2025-meme-bodo (Access on 05.12.2025) 2500 2000 1500 1000 500
0 2000 1500 1000 500 0 Positive
Negative
Neutral
This section discusses the systems utilized by the participants.
• FiRC-NLP [32] explored three distinct approaches: prompt-based inference (Zero-Shot Classification and Few-Shot In-Context Learning via retrieval) using Gemini (Google’s Gemini 2.5 Flash), cross-modality encoders that combine image and text, and a text-only modality leveraging Optical Character Recognition (OCR), OCR-based English translation, and image descriptions. Their final ensemble system fuses these modalities, demonstrating the efectiveness of multimodal fusion and robust training strategies. • CSIS BITS Pilani [33] evaluated multiple dual-encoder architectures, combining the CLIP Vision Transformer with language-specific text models including MuRIL, XLM-Roberta, and M-BERT. Training is conducted using a 5-fold cross-validation strategy with a weighted loss function to counteract class imbalance, and performance is validated on the test set. • Golden Ratio [34] integrated the CLIP vision encoder with tailored Transformer-based language models: MuRIL for Bangla, XLM-RoBERTa for Hindi, and Bodo. Textual and visual features are fused using a cross-attention mechanism to enhance classification accuracy. • SCaLAR [35] experimented with two architectures: a CLIP-based model combining CLIP ViT visual features with multilingual Sentence Transformer embeddings, and a ViT + XLM-Roberta model where visual and textual features were concatenated for classification. The CLIP-based model achieved superior performance in Bangla and Gujarati, underscoring the efectiveness of contrastive pretraining for aligning visual and linguistic representations. • NLPFusion [36] For Bangla memes, authors employed ConvNeXt-small, a state-of-the-art convolutional architecture inspired by transformers, to extract robust semantic visual features. ResNet-34 is used for Hindi and Bodo memes, while ResNet-18 is applied for Gujarati memes. Textual data is processed using transformer-based models with domain-specific pre-training to handle linguistic Rank 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Team FiRC-NLP [32] NLPFusion [36]
MUCS [40]
SCaLAR [35] KK_NLP_AI_IIIT_Ranchi [37]
CSIS BITS Pilani [33]
IReL [42] IIT Dhanbad [38]
CSE_SVNIT [39] YNU (Kongqiang Wang) [41]
VEL (Charmathi Rajkumar) HASOC2025_meme (Baseline)
DeepSemantics [44] HASOC_2025 [43]
diversity and code-mixed content. • KK_NLP_AI_IIIT_Ranchi [37] first experimented with text-only, low-resource models that relied solely on OCR-extracted text. They fine-tuned a multilingual IndicBERT backbone and observed strong performance for Hindi, Gujarati, and Bodo but weak results for Bangla. To investigate whether the poor Bangla performance was due to IndicBERT limitations, they then tested monolingual BERT-based models for each language. These included bengali-bert, hindibert-v2, gujarati-bert, and pretrained-bodo-legal-bert. However, the monolingual backbones did not yield substantial improvements, and in some cases performed worse than the multilingual model. Secondly, They incorporated the CLIP Vision model to combine image features with BERT-based text features using simple feature concatenation. They tested both multilingual and monolingual text backbones with CLIP, finding improvements for Bangla and Gujarati but little to no gain for Hindi and consistently poor results for Bodo. Because CLIP failed to extract meaningful cross-modal features for Indian languages, cross-attention fusion was discarded and only concatenation-based fusion was used. They then evaluated monolingual BERT backbones with CLIP, which further improved performance for Bangla, Gujarati, and Hindi. Finally, due to consistently weak results for Bodo with image features, they submitted a text-only IndicBERT-v2 model for that language. • IIT Dhanbad [38] developed a multimodal deep learning framework for five subtasks—Sentiment, Sarcasm, Vulgarity, Abuse, and Target Communities—to automatically classify hateful and offensive memes. A task-specific strategy was used, where each label was handled by a dedicated multimodal model using various text encoders (XLM-R, mBERT, MuRIL, BanglaBERT) and image encoders (EficientNet, DenseNet, VGG19, ResNet), along with techniques like gated attention and weighted losses. Extensive experiments showed that these task-specific pipelines performed best when combined through ensemble methods. • CSE_SVNIT [39] leveraged pre-trained transformer models including XLM-ROBERTa, IndicBERT, MuRIL, and mBERT enhanced with convolutional neural network (CNN) layers in diferent Indian languages. Experimental results show that MuRIL provides optimal performance and outperforms other transformer models in low-resource Indian languages. • MUCS [40] integrated transformer-based text encoders (Indic-BERT, MuRIL, XLM-Roberta) with convolutional and transformer-based vision models (ResNet, EficientNet, ViT) using two fusion mechanisms -concatenation and attention-based strategies, to efectively capture the complementary cues from both modalities. • HASOC2025_meme (Baseline) system employs zero-shot prompting using the Qwen2-7B-Instruct model 7. Without any task-specific fine-tuning, the model is queried directly with carefully designed prompts to classify memes across the target categories. This baseline demonstrates the efectiveness of lightweight, instruction-tuned language models in handling multimodal meme understanding tasks, especially when annotated training data is limited. • YNU [41] experimented with Gaussian Naive Bayes, Logistic Regression, K-Neighbors Classifier, Support Vector Machine, Decision Tree Classifier, Linear SVC, Random Forest Classifier, later use Ensemble Learning. • IReL [42] developed and evaluated two independent runs under the team name IReL. Run 1 employed XLM-RoBERTa fine-tuned separately for each language (Bangla, Hindi, Gujarati, and Bodo), leveraging multilingual transformer embeddings to capture semantic nuances in noisy meme texts. Run 2 applied a zero-shot approach using ChatGPT specifically for Bodo, addressing the severe lack of annotated resources for this language. Both approaches processed text-based meme content exclusively. 7https://huggingface.co/Qwen/Qwen2-7B-Instruct (Access on 05.12.2025) • HASOC_2025 [43] extracted image features using ResNet-101 and combined them with OCRbased text features from the HASOC-2025 dataset to classify content as abusive/vulgar or not. They addressed class imbalance using ADASYN and trained separate XGBoost classifiers for each language. Their experiments showed that this multimodal approach works well across diferent scripts, even with OCR noise, demonstrating the efectiveness of combining deep image embeddings with traditional machine learning for multilingual hate speech detection. • DeepSemantics [44] introduced a multimodal comparative analysis for detecting content that is, sarcastic, abusive, and vulgar. In addition to that, the sentiment of any content are also captured. The models are applied on the four diferent datasets consisting of diferent languages: Hindi, Bodo, Gujarati, and Bengali. Experiments are performed on various models, of which VisualBert was found to give the best performance. This model is most efective for detecting hateful and ofensive content in multilingual datasets. • FAST [46] used (i) a tailored preprocessing pipeline for noisy OCR and Hindi, English code-mixing using curated stopword and vulgar dictionaries, (ii) a combination of lightweight classical models (TF–IDF + Random Forest) with neural approaches (CNN, BiLSTM, ResNet50). • CNLP-UPES [45] proposed a multimodal framework that integrates BERT-based OCR text embeddings and ResNet-derived image features through an early-fusion strategy, enabling multi-task predictions across five classification subtasks. To address class imbalance, we incorporate RandomOverSampler, thereby enhancing the representational balance of minority categories.
In this work, we presented the results of the HASOC Meme 2025 shared task on abusive meme identification across four languages: Bangla, Hindi, Gujarati, and Bodo. The task attracted a significant number of participants, reflecting the growing research interest in multimodal hate and abuse detection for Indian languages. Teams explored a wide range of approaches, including Zero-Shot Classification, Few-Shot In-Context Learning with retrieval-based prompting, and CLIP-based Vision Transformers, demonstrating the rapid evolution of multilingual and multimodal learning techniques.
While the shared task yielded promising insights, it also highlighted substantial gaps, especially for low-resource languages. Further experimentation is essential, particularly in building richer multimodal datasets, improving cross-lingual transfer, and developing robust models capable of handling linguistic diversity and culturally grounded abusive content. Continued eforts in these directions will enable more inclusive and efective solutions for combating harmful content across a broader spectrum of languages.
We thank Mr. Atri Chandra, Mr. Farhan Chowdhury, Mr. Animesh Howladar, Mr. Suvankar Dey, Mr. Trideep Ghosh, Mr. Alankrita Kumari, Mr. Ayush Lodh, Mr. Priyanka Das, Ms. Aveepsa Hatua, Mr. Bhaskar Pal, Mr. Gajjar Kandarp Dipakkumar, Ms. Vanpariya Palak Govindbhai, Mr. Patel Vraj Bipinbhai, Ms. Nijira Mushahary for the dataset annotation. We also thank the FIRE and HASOC organizers for their support in organizing the track. We thank all participants for their submissions and their valuable work.
During the preparation of this work, the author(s) used Grammarly in order to: Grammar and spelling check. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content. [30] G. Kumari, A. Sinha, A. Ekbal, A. Chatterjee, V. B. N, Enhancing the fairness of ofensive memes detection models by mitigating unintended political bias, J. Intell. Inf. Syst. 62 (2024) 735–763.
URL: https://doi.org/10.1007/s10844-023-00834-9. doi:10.1007/s10844- 023- 00834- 9. [31] S. Suryawanshi, B. R. Chakravarthi, P. Verma, M. Arcan, J. P. McCrae, P. Buitelaar, A dataset for troll classification of TamilMemes, in: G. N. Jha, K. Bali, S. L., S. S. Agrawal, A. K. Ojha (Eds.), Proceedings of the WILDRE5– 5th Workshop on Indian Language Data: Resources and Evaluation, European Language Resources Association (ELRA), Marseille, France, 2020, pp. 7–13.
URL: https://aclanthology.org/2020.wildre-1.2/. [32] F. Hassan, M. S. Jahan, E. Migaev, F. Mtumbuka, Bridging modalities for hate speech detection in memes, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [33] R. Bohra, Y. Sharma, A multi-modal ensemble approach for hate speech and ofensive content detection in indic memes, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [34] T. Paul, A. Jamatia, Hate speech and ofensive content identification in memes in bangla, hindi, and bodo, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [35] S. S. Nandam, A. K. Madasamy, Multimodal hate speech and ofensive content classification in code-mixed indian language memes, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [36] S. Coelho, A. Hegde, A. M Shetty, Nitte, Hasoc-meme: Enhancing hate speech recognition in bengali, hindi, gujarati, and bodo memes using multimodal multitask transformers, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [37] U. Kedia, S. Prakash, K. Kumari, K. Prakash, T. Kumar, A transformer-based approach to multimodal hateful meme classification, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [38] T. Kumari, A. Das, A. Sarvaiya, Hateful and ofensive meme detection in multimodal memes dataset for indo-aryan languages, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [39] S. S. Sahu, J. Damor, A transformer-based model for hate speech detection in low resource indian languages, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [40] R. B N, S. H L, Towards safer social media: Multimodal hate speech detection in memes across diverse indian languages, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [41] W. Kongqiang, T. Qingli, Hate speech and ofensive content identification in memes in bangla, hindi and gujarati using auxiliary text supervised learning, in: Working Notes of FIRE 2025 Forum for Information Retrieval Evaluation, CEUR, 2025. [42] K. Tewari, S. Chanda, A. Namdeo, S. Pal, Savior: Sentiment, sarcasm, abuse, and vulgarity in online realities (memes), in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [43] S. Singh, G. Kumar, J. P. Singh, S. K. Rai, K. Goswami, Multilingual hate speech classification in memes using ocr-extracted text and visual features, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [44] A. Malviya, A. Basak, S. Choudhury, Fusionguard: Visual-linguistic representations for multilingual harmful content detection, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [45] P. Dadure, S. Ghosh, Bodo meme classification using multimodal fusion and oversampling in hasoc-2025, in: Working Notes of FIRE 2025 - Forum for Information Retrieval Evaluation, CEUR, 2025. [46] M. Rafi, Fast-hasoc 2025: Multimodal and multilingual approaches for hate speech and ofensive content detection in hindi memes, in: Working Notes of FIRE 2025 - Forum for Information