The widespread adoption of Large Language Models (LLMs) has introduced new risks associated with the fluent generation of factually incorrect or misleading content. Addressing this challenge requires fine-grained tools capable of not only detecting misinformation but also distinguishing among types of factual errors. Prompt RecOvery for MisInformation Detection (PROMID)-2025 shared task, held as part of the Forum for Information Retrieval Evaluation (FIRE)-2025, directly targets this issue by encouraging systems that analyze and classify incorrectness in LLM-generated outputs. We - team MUCS participated in Subtask 2, which requires classifying LLM-generated summaries into one of four predefined misinformation categories: misrepresentation, fabrication, false attribution, and incorrect quantities. The subtask is challenging due to the semantic similarity between classes and the multilingual setting that includes Kannada, Malayalam, Tamil, and Telugu. We propose three multilingual Deep Learning (DL) pipelines: (i) Bidirectional Long Short Term Memory (BiLSTM) model, (ii) Transformer + BiLSTM hybrid model, and (iii) Bidirectional Gated Recurrent Unit (BiGRU) model, to categorize each data point into one of the four diferent categories based on the presence of factual incorrectness in the summaries. Each model employs language-aware pre-processing, subword-aware tokenization, and contextual encoders tailored to sequence modeling. The Transformer + BiLSTM model integrates transformer encoders, BiLSTM layers, and multi-head self-attention to capture both global and local dependencies. In contrast, BiLSTM and BiGRU models use simpler recurrent architectures combined with attention mechanisms to reduce computational overhead. To address mild class imbalance, we apply focal loss during training along with mixed-precision optimization for eficiency. The proposed models obtained best performances with: BiLSTM model ranking 1 st in both Tamil and Telugu, BiGRU model ranking 2nd in Kannada, and the Transformer + BiLSTM model ranking top-3 position in Malayalam. These results demonstrate the utility of hybrid neural modeling and linguistically-informed pre-processing for multilingual misinformation classification in LLM-generated content.
The emergence of powerful LLMs such as Generative Pre-trained Transformer (GPT)-4, Pathways
Language Model (PaLM), and Large Language Model Meta AI (LLaMA) has significantly transformed
the landscape of natural language generation. These models exhibit unprecedented fluency, contextual
awareness, and language understanding across a wide range of tasks. However, their susceptibility to
generate factually incorrect outputs—commonly referred to as hallucinations, has become a matter of
growing concern [
Combating misinformation in LLM-generated content is not only a matter of detecting falsehoods but
also of understanding their origin. Many instances of misinformation can be traced back to ambiguous or
misleading prompts that guide the model to generate specific types of incorrect summaries. Traditional
misinformation detection systems often neglect this generative aspect, focusing on surface-level textual
features or post-analysis of static web content [
We, team MUCS, address the challenges of Subtask 2 by designing a language-agnostic misinformation classification framework capable of capturing both contextual semantics and syntactic nuances that diferentiate hallucinated content types. We develop three end-to-end multilingual DL pipelines: (i) BiLSTM model, (ii) Transformer + BiLSTM hybrid model, and (iii) BiGRU model, to categorize each data point into one of the four diferent categories based on the presence of factual incorrectness in the summaries. Each pipeline incorporates robust text pre-processing, custom subword-aware tokenization, and neural encoders designed to classify misinformation into one of four predefined categories across four South Indian languages - Kannada, Malayalam, Tamil, and Telugu. The architectures vary in complexity: the Transformer+ BiLSTM model includes transformer encoders, multi-head self-attention, and BiLSTM layers, while the BiLSTM and BiGRU models use purely recurrent layers along with attention mechanisms tailored to their depth. These configurations allow the models to learn both global dependencies and local sequential patterns—critical for distinguishing subtle misinformation cues. To address the mild class imbalance present in the dataset, we apply the focal loss function with class-aware weighting. Additionally, mixed-precision training is used for computational eficiency in deeper configurations.
Performances of the proposed models varied across languages depending on the architecture. The
BiLSTM model attained Rank 1 in both Tamil and Telugu, the BiGRU model achieved Rank 2 in Kannada,
and the Transformer + BiLSTM model secured a consistent top-3 position in Malayalam. These results
underscore the benefits of combining linguistic preconditioning, cross-lingual representation learning,
and architecture-specific modeling choices for classifying misinformation. Overall, this work contributes
to the broader goal of developing robust and prompt-resilient misinformation detection systems for
LLM-generated content [
The subsequent sections of this paper details the related works (Section 2), methodology (Section 3), experiments, results, and implications of our approach (Section 4), followed by conclusion and future works (Section 5).
Recent advancements in natural language generation have intensified eforts to detect hallucinated or misleading content automatically. Misinformation classification, particularly in LLM-generated text, intersects with factuality evaluation, hallucination detection, prompt sensitivity, and deep contextual modeling. Researchers have proposed a broad spectrum of models, evaluation metrics, and benchmarks to address these challenges, ranging from Question Answer (QA) based fidelity evaluation to fine-grained hallucination categorization.
Ji et al. [
Manakul and Gales [
Gupta and Vishwakarma [
Alam et al. [
Razeghi et al. [
Collectively, these studies ofer strong foundational insights into hallucination detection and
misinformation classification. Yet, challenges remain with respect to multilingual support, hallucination
attribution granularity, prompt variability, and label-level explainability. Although SelfCheckGPT and
FactScore contribute to progress in zero-reference evaluation, and taxonomy eforts enhance
annotation clarity [
This section describes the three proposed multilingual DL pipelines. Each pipeline processes summaries and source articles in South Indian languages - Kannada, Malayalam, Tamil, and Telugu, and assigns one of the four misinformation class labels to the given input. The models difer in tokenization, sequence encoders, attention mechanisms, and optimization strategies. Each pipeline follows a general structure of: (i) pre-processing the input, (ii) tokenization, (iii) sequence encoding with neural layers, and (iv) classification. The proposed end-to-end multilingual DL pipelines for Subtask 2 is shown in Figure 1 and a description of the models is given in the following sub-sections.
The BiLSTM model serves as a strong baseline in our system. It combines a deep recurrent encoder with multi-head self-attention to model sequential dependencies and incorporate global context for efective hallucination detection. The details of the model are given below: • Text Pre-processing: – URLs and email address are removed using regex1. – Punctuation characters (e.g., !"#$...~) are stripped. – Whitespace is normalized by collapsing multiple spaces and trimming. – No long-token removal is used in this basic version. – We employ a BasicTokenizer that splits input text on whitespace characters. – Language indicator tokens (<ta>, <kn>) are retained. – Tokens longer than 10 characters are split into fixed-length, non-overlapping 5-character chunks2. 1https://en.wikipedia.org/wiki/Uniform_Resource_Locator 2Inspired by Byte-Pair Encoding (BPE) principles: https://huggingface.co/blog/how-to-preprocess
– Vocabulary is created using tokens with a frequency ≥2 in the training set. • Model and Training Loop: – The architecture is based on BiLSTM3 - a recurrent model that captures both past and future context by processing the sequence in forward and backward directions. – Each token is embedded into a 256-dimensional vector. – A linear layer with dropout simulates positional information. – A 2-layer BiLSTM with 512 hidden units per direction outputs 1024 dimensional contextual embeddings. – A Multi-Head Self-Attention layer4 with 8 heads learns token importance for downstream aggregation. 3https://pytorch.org/docs/stable/generated/torch.nn.LSTM.html 4https://pytorch.org/docs/stable/generated/torch.nn.MultiheadAttention.html – Masked average pooling skips padding and compress the sequence into a fixed-length vector. – A Multi-Layer Perceptron (MLP) classifier with dimensions 1024 → 512 → 256 → 4 computes the class prediction. MLPs are fully connected layers that use GELU activations [17], LayerNorm, and dropout for regularization. – We apply Weighted Cross-Entropy Loss where class weights are computed as the inverse frequency from the training labels. – Optimization uses the AdamW optimizer [18] — an adaptive gradient method with decoupled weight decay. – Learning rate is controlled via the OneCycleLR scheduler [19], which warms up and gradually cools down the learning rate for better convergence. – Gradient clipping is applied to prevent exploding gradients.
– Training is conducted on mini-batches (size 16) for 10 epochs.
This model provides a balance between performance and computational simplicity, ofering strong results across two languages while using a purely recurrent encoder architecture. Its relatively lightweight design makes it particularly efective for resource-constrained settings.
This hybrid model combines a Transformer Encoder with a BiLSTM network, enabling a fusion of global attention-based contextual modeling and sequential dependency learning. It is the most architecturally complex of the three approaches. The details of the model are given below: • Text Pre-processing: • Tokenization: • Model and Training Loop: – Same as in BiLSTM model, but includes a filter to discard tokens with length ≥50 5. – The EnhancedTokenizer splits on whitespace and segments tokens >10 characters using overlapping 5-character windows with stride 3. – This overlapping subword approach captures richer morphological structures for agglutinative languages. – Vocabulary is constructed with min_freq = 2. – Tokens are mapped to 384 dimensional embeddings plus learned positional embeddings. – A 3-layer Transformer Encoder applies multi-head self-attention and feed-forward layers to model global context. – The two representations (Transformer + BiLSTM) are concatenated and projected down to 768-d using a linear layer. – Outputs are passed to a 2-layer BiLSTM (384 hidden units per direction). – After dropout and LayerNorm, masked average pooling yields a fixed vector.
– An MLP classifier (768 → 384 → 4) predicts the class. • Loss and Optimization: – We use FocalLoss [20] with focusing parameter = 2.0 and class weighting . – It improves learning by penalizing easy examples and focusing on hard-to-classify ones. – Optimizer: AdamW with weight decay.
– Scheduler: OneCycleLR with warm-up phase. 5Extreme-length tokens often indicate noise from LLM-generated or malformed inputs – Mixed-precision training is enabled via Automatic Mixed Precision (AMP)6.
– Gradient accumulation is used to emulate larger batch sizes.
This architecture benefits from the transformer’s ability to capture long-range dependencies and the BiLSTM’s ability to maintain sequential coherence. It performed robustly on morphologically rich scripts, demonstrating its suitability for complex linguistic structures.
The BiGRU model is a lightweight architecture that discards transformer components in favor of a purely recurrent encoder with attention. It serves as a strong baseline in low-resource scenarios. The details of the model are given below: – Same as in Transformer + BiLSTM model and URL + punctuation removal, space normalization, and dropping tokens >50 characters. – SimpleTokenizer applies minimal whitespace-based splitting. – No subword segmentation is used (baseline). – 256 dimension tokens processed by a 2-layer Bidirectional GRU (BiGRU)7 with 512 units per direction. – Additive attention computes token-level importance using: Linear(1024 → 512) → Tanh → Linear(512 → 1) followed by Softmax. – A weighted sum over token vectors produces a document representation.
– MLP classifier: 1024 → 512 → 256 → 4. • Loss and Optimization: – FocalLoss with = 2.0 and class-weighted . – AdamW optimizer. – CosineAnnealingLR scheduler8 reduces learning rate smoothly.
– Model trained with batch size 32; best checkpoint chosen based on lowest loss.
Due to its simplicity, fast convergence, and minimal memory overhead, the BiGRU model is ideal for deployment in real-time applications or where computational resources are limited.
The configuration of hyperparameters used in training these three models are provided in Table 1.
This section presents the empirical evaluation of our proposed system on Subtask 2 of the PROMID-2025 shared task. We report both quantitative performance scores and qualitative insights. The evaluation is designed to assess the system’s generalization across diverse linguistic structures in four Indian languages: Kannada, Malayalam, Tamil, and Telugu. Each model is evaluated using Precision, Recall, Macro F1-score, and overall classification Accuracy. A description of the dataset, followed by a detailed analysis of model performance for each language, is provided below: 6https://pytorch.org/docs/stable/amp.html 7https://pytorch.org/docs/stable/generated/torch.nn.GRU.html 8https://pytorch.org/docs/stable/generated/torch.optim.lr_scheduler.CosineAnnealingLR.html
Description A short title or headline summarizing the article topic Additional or extended versions of the title, if available The full ground-truth article body used as the reference source A hallucinated or misleading summary, typically generated by an LLM One of four misinformation categories: misrepresentation, fabrication, false attribution, or incorrect quantities
A human-written, factually accurate summary of the article
The multilingual misinformation classification dataset [ 21, 22] released as part of the PROMID-2025 [23] shared task contains hallucinated summaries generated by LLMs paired with gold-standard articles in four South Indian languages: Kannada, Malayalam, Tamil, and Telugu. Each summary is labeled with one of four incorrectness types: misrepresentation, fabrication, false attribution, and incorrect quantities. The overall category-level distribution across all training data is as follows: • Misrepresentation: 29.7% • Fabrication: 25.3% • False Attribution: 25.3% • Incorrect Quantities: 19.7% Table 2 gives the description of the datasets and Table 3 shows the class-wise distribution.
The models are evaluated using four standard metrics: macro-averaged Precision, Recall, F1-score, and overall Accuracy, and ranked based on macro-averaged F1-score. We tested the proposed multilingual models: BiLSTM model, Transformer + BiLSTM hybrid model, and BiGRU-based model, on each language. Table 4 presents the performance metrics for each model across the four languages and Figure 2 ofers a visual comparison of team-wise macro F1-scores of all the participating teams in Subtask 2 across the four languages.
Our analysis reveals that the BiLSTM model shows illustrated performance for Tamil and Telugu, achieving the highest F1-scores and accuracies. This suggests that simpler architectures with adequate attention mechanisms, when paired with class-weighted loss, can efectively capture essential patterns in morphologically rich languages. For Kannada, the BiLSTM model attains the best F1-score, whereas the BiGRU model yields the highest accuracy. This variation suggests that while BiGRU may be better at general predictions, BiLSTM model maintains precision-recall balance better. For Malayalam, the BiGRU model outperforms both the BiLSTM and Transformer + BiLSTM variants across all metrics demonstrating that lightweight architectures can still perform competitively for certain languages.
Although the dataset is relatively balanced overall, some class imbalance particularly under the incorrect quantities category introduces challenges. Combined with semantic overlap between categories such as false attribution and fabrication, this afects low-recall cases. Performance diferences also underline the influence of architecture depth, loss strategy, and attention mechanisms. The use of focal loss with class-aware weighting and sequence-level attention improves outcome reliability, especially in low-resource, multilingual settings.
Our multilingual DL system for Subtask 2 of the PROMID-2025 shared task demonstrates strong performance in classifying LLM-generated summaries into fine-grained misinformation categories across four South Indian languages—Kannada, Malayalam, Tamil, and Telugu. Leveraging hybrid transformerrecurrent architectures, attention-based encoding, and class-aware focal loss, the Transformer + BiLSTM model efectively captures both sequential and contextual signals required to detect subtle factual inconsistencies. While the attention mechanism and focal loss are employed across all three models, the hybrid architecture most directly benefits from combined transformer and recurrent components. BiLSTM model achieved Rank 1 in Telugu and Tamil, BiGRU model achieved Rank 2 in Kannada, and Transformer + BiLSTM model obtained Rank 3 in Malayalam, based on macro F1-scores submitted to the oficial leader board. These results reflect not only the adaptability of our architecture across typologically distinct languages but also the efectiveness of pre-processing strategies, subword-aware (a) Kannada
(b) Malayalam (c) Tamil (d) Telugu tokenization, and stable training under a mildly imbalanced class distribution. To further improve the system, we plan to explore methods for enhancing model interpretability, particularly in generating explanations for the predicted misinformation types. We also intend to evaluate lightweight model variants for more eficient deployment in practical applications. These directions are aimed at making LLM-based misinformation detection more scalable, interpretable, and robust in multilingual scenarios.
In the course of preparing this paper, we made limited use of a generative AI assistant to support the writing process. The tool was used primarily for language refinement, section structuring, and LaTeX formatting consistency. All technical content, including experimental design, model implementation, and results, was conceived, executed, and validated entirely by the authors. The AI assistant did not contribute novel research ideas, nor did it influence the reported findings. Its role was strictly supportive comparable to using grammar checkers or typesetting tools and all content included in this manuscript has been carefully reviewed and approved by the authors. [17] D. Hendrycks, K. Gimpel, Gaussian Error Linear Units (GELUs), arXiv preprint arXiv:1606.08415 (2016). URL: https://arxiv.org/abs/1606.08415. [18] I. Loshchilov, F. Hutter, Decoupled Weight Decay Regularization, 2019. URL: https://arxiv.org/abs/ 1711.05101. arXiv:1711.05101. [19] L. N. Smith, A Disciplined Approach to Neural Network Hyper-parameters: Part 1 – Learning Rate, Batch Size, Momentum, and Weight Decay, 2018. URL: https://arxiv.org/abs/1803.09820. arXiv:1803.09820. [20] T.-Y. Lin, P. Goyal, R. Girshick, K. He, P. Dollár, Focal Loss for Dense Object Detection, in: Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017, pp. 2999– 3007. doi:10.1109/ICCV.2017.324. [21] G. K. Shahi, Y. Mejova, Too Little, Too Late: Moderation of Misinformation around the RussoUkrainian Conflict, in: Proceedings of the 17th ACM Web Science Conference (WebSci ’25), Association for Computing Machinery, 2025. doi:10.1145/3717867.3717876. [22] G. K. Shahi, T. A. Majchrzak, AMUSED: An Annotation Framework of Multimodal Social Media Data, in: F. Sanfilippo, O.-C. Granmo, S. Y. Yayilgan, I. S. Bajwa (Eds.), Intelligent Technologies and Applications, Springer International Publishing, Cham, 2022, pp. 287–299. [23] S. Satapara, P. Mehta, D. Ganguly, S. Modha, Fighting Fire with Fire: Adversarial Prompting to Generate a Misinformation Detection Dataset, CoRR abs/2401.04481 (2024). URL: https://doi.org/ 10.48550/arXiv.2401.04481. doi:10.48550/ARXIV.2401.04481. arXiv:2401.04481.