This paper presents the work of the MMA team for Task 2 - Claim Normalization of the CheckThat! 2025 shared task. The task aims to convert noisy and informal social media posts into concise, check-worthy claims suitable for fact-checking. Our experiments focused on the monolingual setup of the task. While we submitted runs for all languages provided in the task, we provided additional experiments tailored for Arabic. We explored a range of approaches, including T5-based sequence-to-sequence models, zero-shot prompting, fine-tuning LLMs, and data augmentation techniques. The best-performing configurations were selected for each language. In particular, our Arabic model, using umt5 with data augmentation, achieved a strong score of 0.4584, placing third among all submissions. Spanish achieved the highest score of 0.5094 with the base umt5 model, while languages such as Polish and German showed lower performance. These results demonstrate the efectiveness of our multilingual strategies and the impact of data augmentation in improving performance, particularly for low-resource languages.
The digital age has transformed social media platforms into primary channels for information. However,
this rapid and unregulated flow of content has facilitated the widespread circulation of false or misleading
information shared without malicious intent [
To mitigate previous problems, researchers are developing automated fact-checking systems and
collaborative verification methods to address the challenges posed by the diverse formats, varying
source credibility, and complex user interactions in online environments [
In this study, we present our approach for Task 2 of the CLEF2025 CheckThat! Lab [
To better understand how to extract and normalize social media claims, we look at related work in four main areas. First, we explain how claims are defined and show the importance of normalization. Second, we explore how sequence-to-sequence models help generate claims from longer posts. Then, we discuss how data augmentation can improve model performance when training data is limited. Finally, we investigate the use of large language models (LLMs) to perform claim extraction and normalization more efectively even with little or no fine-tuning.
A claim in the context of fact-checking is typically defined as a declarative statement asserting a piece
of information as true, which can be subsequently verified for its accuracy [
Claim normalization refers to the process of transforming an extracted claim into a standardized,
concise, and context-independent statement that is suitable for verification [
Sequence-to-sequence (Seq2Seq) models, particularly those based on the Transformer architecture
[
Several studies have demonstrated the efectiveness of Transformer-based models like BART [
The performance of deep learning models, including Seq2Seq Transformers, is heavily dependent on
the availability of large, high-quality training datasets. For specialized tasks like claim extraction from
social media, particularly in multiple languages, such datasets are often scarce or expensive to create.
Data augmentation techniques are employed to artificially expand the training set, thereby improving
model generalization and robustness [
Common NLP data augmentation methods include back-translation, translating a sentence to a
target language and then back to the original, synonym replacement, and paraphrasing [
Recent advancements in Large Language Models (LLMs), such as GPT-3.5, GPT-4 [
Moreover, the inherent reasoning abilities of LLMs have been harnessed to improve the accuracy of
claim verification. Kojima et al. [
In the realm of claim extraction, LLMs have demonstrated proficiency in identifying and structuring
claims from unstructured text. For instance, Liu et al. [
We investigate multilingual fine-tuning with T5-based architectures, zero-shot and fine-tuned large language models, and data augmentation techniques. Each approach is described in detail in the following subsections. The quantitative analysis and comparison between diferent approaches are given in section 4.
We adopted the google/umt5 model[25], a multilingual version of T5 designed for cross-lingual generation tasks. We fine-tuned it for post-to-claim generation and explored two training configurations: • Unified Multilingual Training : We trained a single umt5 model on the entire multilingual dataset. This approach leverages shared cross-lingual representations, enabling low-resource languages to benefit from richer ones through transfer learning. • Language-Specific Training : We trained separate umt5 models per language. While this setup goes beyond cross-lingual transfer, it allows each model to better specialize in the linguistic nuances of its respective language, potentially improving claim generation fidelity.
We evaluated the generative capability of Qwen 2.5 [
You must filter the noise from the post.
You must respond with the same language of the post.
Ignore repetition, rhetorical flourishes, and background anecdotes.
Keep only the essential elements: • Who is involved (names, age, nationality). • What is being claimed will happen or has happened. • Where/When only if they are integral to the claim.
The claim must only include words from the post and do not use any external words. The claim must be in the style of title for an article.
The claim must be Declarative Short Sentence with preserving names and places.
DO NOT add, omit, or translate information. Preserve names and wording where possible.
To improve the performance of the Qwen2.5-7B model, we applied parameter-eficient fine-tuning using Low-Rank Adaptation (LoRA)[26]. The training pipeline involved a two-stage strategy: 1. Key Point Extraction: Using Qwen2.5-32B in a zero-shot setting, we extracted key points from each training example (post and normalized claim) with the following system instruction:
- Identify and extract the important key points presented in this post. - Then, you have to use all these key points to extract the main claim of that post. You must answer in the same language as the given post. The output must be in the following format: <keypoints> [’key1’,’key2’,....] </keypoints> <claim> claim of the post </claim>
2. Claim Generation from Key Points: We fine-tuned Qwen2.5-7B model on this augmented dataset to map input posts to final claims via the extracted key points. The objective was to focus the model’s attention on important information and reduce the noise from irrelevant content.
While previous sections focused on multilingual and monolingual experiments across 13 languages, we dedicated additional efort to improving performance on Arabic, given its linguistic complexity. This involved both model selection and targeted data augmentation tailored to Arabic content.
We experimented with the following variants of T5 models, specifically ara-t5 models [ 27], specialized for Arabic: • ara-t5-v2, a general-purpose Arabic T5 model, • ara-t5, trained for title generation tasks, • ara-t5-tweet, fine-tuned on Arabic tweet data.
In addition to modeling variations, we enriched the Arabic training data using the Google Fact Check Tools API [28]. This API provides access to a curated set of verified claims and related content. We scraped Arabic post-claim pairs from this resource and incorporated them into our training pipeline. The goal was to increase the amount of high-quality, claim-specific data for Arabic, which is typically underrepresented.
These augmented samples were used to fine-tune the multilingual umt5, which showed superior performance and was ultimately selected for final submission. An example of the augmented Arabic data is shown below:
Ahﻧأ ﺖﻋدا wﻳdyﻓ ﻊVAqﻣو ًاCw} ﻲﻋAmtﺟﻻا ﻞ}اwtﻟا ﻊﻗاwﻣ Ylﻋ ‹AﺑAsﺣو ‹Aﺤf} ﺖﻟواdﺗ ةrybﻛ ‹Aymﻛ Ahyﻓ rh\ﺗو ,Tnsﻟا xأC Tlfﺣ ءAhtﻧا dﻌﺑ مwyﻟا حAb} HﻳCAﺑ عCاwJ rh\uﺗ Yﻟإ دwﻌﺗ Tﻟواdtmﻟا wﻳdyfﻟا ﻊVAqﻣو CwOﻟAﻓ ,ﺢyﺤ} ryﻏ ءAﻋدﻻا ا@ﻫ نأ ﻻإ .‹AﻳAfnﻟا ﻦﻣ دwﻌﻳ AhSﻌﺑ .Tnsﻟا xأC ‹ﻻAftﺣا dﻌﺑ HﻳCAbﻟ ﺖsyﻟو ,Tfltﺨﻣ ﺦﻳCاwﺗ ﻲﻓو ىrﺧأ ﻦﻛAﻣأ .Ahsfﻧ HﻳCAﺑ ﻲﻓ TqﺑAF ‹ﻻAftﺣا ﻦﻣ rﺧﻵا AhSﻌﺑو ,AyﻟAWﻳإ ﻲﻓ ﻲﻟwﺑAﻧ Yﻟإ Claim: xأC Tlfﺣ dﻌﺑ HﻳCAﺑ عCاwJ rh\uﺗ Ahﻧأ ﻲﻋdﺗ ﻲtﻟا Tﻟواdtmﻟا wﻳdyfﻟا ﻊVAqﻣو CwOﻟا .ىrﺧأ ﻦﻛAﻣأ ﻦﻣ وأ Tmﻳdﻗ wﻳdyﻓ ﻊVAqﻣو Cw} ﻲﻫ Tnsﻟا
For our experiments, we utilized the dataset provided by the CheckThat! 2025 Task 2: Claim
Normalization [
Our participation was exclusively in the monolingual setup, focusing on languages with complete datasets. Table 1 presents the aggregated statistics of the dataset across the training, development, and test splits for these 13 languages.
For evaluation, we employed the METEOR score [29], as specified by the task organizers. METEOR is designed to evaluate the quality of machine-generated text by comparing it to reference texts. It calculates a harmonic mean of unigram precision and recall, with recall weighted higher than precision. Additionally, METEOR incorporates features such as stemming, synonymy matching, and a fragmentation penalty to account for word order, making it more aligned with human judgment.
We conducted our experiments using a single NVIDIA A6000 GPU. Our training encompassed two primary approaches: fine-tuning T5-based models and fine-tuning large language models (LLMs) using Low-Rank Adaptation (LoRA). 4.2.1. T5-Based Models For fine-tuning T5-based models, we employed the following hyperparameters: • Number of epochs: 20 • Learning rate: 5e-4 • Learning rate scheduler: Cosine with 90 warmup steps • Optimizer: AdamW • Weight decay: 0.01 • Efective batch size : 32
During evaluation, we selected the best checkpoint and then utilized the following decoding parameters: • Maximum sequence length: 512 • Number of beams: 5 • Top-p: 0.85 • Top-k: 40 4.2.2. Large Language Models with LoRA For fine-tuning large language models using LoRA, we adopted a parameter-eficient approach, adjusting only a subset of the model’s parameters. The hyperparameters for this setup were: • Number of epochs: 5 • Learning rate: 1e-5 • Efective batch size : 8 • LoRA rank: 8
We conducted a comprehensive evaluation across multiple training paradigms to assess the impact of multilingual modeling, zero-shot generalization, and parameter-eficient fine-tuning. Our experiments use the METEOR score as the evaluation metric across 13 languages in the monolingual setting. Multilingual vs. Language-Specific Models. We first trained a single umt5 model jointly on all 13 languages. While this setup benefits from shared multilingual patterns, it showed limitations in capturing language-specific nuances. To address this, we trained a separate umt5 model for each language. As shown in Table 2, language-specific models outperformed the multilingual one in several languages. However, in a few cases like Marathi and Punjabi, the multilingual model demonstrated better performance, likely due to limited monolingual data for those languages. Since the monolingual setup outperformed the multilingual one in most cases and is better suited to capturing the linguistic characteristics of each language, we adopted it for our final submission across all languages—except for Arabic, where the multilingual umt5 model with data augmentation achieved superior results. Zero-Shot Inference with LLMs. We evaluated the zero-shot capabilities of two large language models Qwen2.5-7B and Qwen2.5-32B. As expected, the 32B variant achieved better performance across almost all languages, highlighting the positive correlation between model size and generalization in zero-shot settings. However, variants lagged behind fine-tuned models, especially in low-resource languages like Hindi and Punjabi.
Parameter-Eficient Fine-Tuning. The results were competitive, achieving an average improvement over the zero-shot 7B baseline. For example, in Tamil, LoRA fine-tuning reached 0.437 compared to 0.156 in zero-shot 7B and 0.295 in zero-shot 32B. This demonstrates that parameter-eficient tuning can significantly close the gap to full fine-tuning with much lower resource requirements. Arabic-Specific Modeling. In addition to our multilingual experiments, we performed a focused evaluation on Arabic using various variants of the AraT5 model to evaluate their suitability for claim normalization.
Their METEOR scores on the development and the test sets are shown in Table 3. Among them, ara-t5-v2 yielded the best performance. However, despite promising results, these models were outperformed by the multilingual umt5, especially when combined with data augmentation strategies.
To further boost performance, we extended the training data using scraping post-claim examples using the Google Fact Check Tools API. As shown in Table 4, the best result (0.4584 on the test set) was obtained using umt5 trained with the full augmented dataset, confirming the advantage of cross-lingual modeling with targeted data enhancement.
Final Results on the Test Set. Table 4 presents our final submitted results along with the final rank on the leaderboard. Due to evaluation issues with the Thai language, we submitted predictions for 12 out of the 13 target languages. For each language, the best-performing configuration was selected. The Arabic model achieved a strong score of 0.4584 using umt5 with targeted data augmentation, while Spanish attained the highest overall score of 0.5094 with the base umt5 model. On the other hand, languages such as Polish and German yielded comparatively lower scores, suggesting the need for additional data augmentation or model adaptation to improve performance.
UMT5 (All) UMT5 (Mono) Qwen2.5-7B Zero-shot Qwen2.5-32B Zero-shot Qwen2.5-7B-LoRA
In this paper, we presented our contribution to Task 2 of the CLEF2025 CheckThat! Lab, addressing the multilingual challenge of claim extraction and normalization from social media content. We developed transformer-based models, primarily focusing on the monolingual setup across 13 languages. Our approach combined multilingual T5 variants, zero-shot prompting of large language models, and parameter-eficient fine-tuning with LoRA, alongside a novel keypoint-guided generation strategy. Data augmentation, especially via the Google Fact Check Tools API, played a vital role in improving model performance in low-resource settings, e.g., for Arabic language. Our experiments demonstrated the efectiveness of these methods, with umt5 models and keypoint-assisted strategies yielding strong results. While our work is limited to the monolingual track, future eforts will target improved keypoint generation and broader support for zero-shot settings. Our findings highlight the potential of tailored multilingual systems in enhancing claim verification pipelines and combating misinformation online.
During the preparation of this work, we used ChatGPT and Grammarly for grammar and spelling checks, as well as for paraphrasing and rewording. After using these tools, we carefully reviewed and edited the content as needed and take full responsibility for the final version of this publication. ifndings-emnlp.769/. doi: 10.18653/v1/2024.findings-emnlp.769. [24] M. Sundriyal, T. Chakraborty, P. Nakov, From chaos to clarity: Claim normalization to empower fact-checking, 2024. URL: https://arxiv.org/abs/2310.14338. arXiv:2310.14338. [25] H. W. Chung, X. Garcia, A. Roberts, Y. Tay, O. Firat, S. Narang, N. Constant, Unimax: Fairer and more efective language sampling for large-scale multilingual pretraining, in: The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023, OpenReview.net, 2023. URL: https://openreview.net/forum?id=kXwdL1cWOAi. [26] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, LoRA: Low-rank adaptation of large language models, in: International Conference on Learning Representations, 2022. URL: https://openreview.net/forum?id=nZeVKeeFYf9. [27] E. M. B. Nagoudi, A. Elmadany, M. Abdul-Mageed, AraT5: Text-to-text transformers for Arabic language generation, in: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, Dublin, Ireland, 2022, pp. 628–647. URL: https://aclanthology.org/2022.acl-long.47. [28] Google fact check tools, 2024. https://toolbox.google.com/factcheck/explorer. [29] A. Lavie, A. Agarwal, Meteor: an automatic metric for mt evaluation with high levels of correlation with human judgments, in: Proceedings of the Second Workshop on Statistical Machine Translation, StatMT ’07, Association for Computational Linguistics, USA, 2007, p. 228–231.