We present an overview of the CheckThat! Lab 2025 Task 3, part of CLEF 2025. The task focuses on verifying claims with numerical quantities and temporal expressions. Numerical claims are defined as those requiring validation of explicit or implicit quantitative or temporal details. It is conducted in three languages: Arabic, Spanish, and English. A total of 258 valid runs were submitted by 13 unique teams across languages, with 4 participants in Spanish and Arabic. 10 teams participated in fact-checking English numerical claims. Among these teams, the use of transformer pre-trained language models (PLMs) was the most frequent. A few teams also employed Large Language Models (LLMs). We provide a description of the dataset, the task setup, including evaluation settings, and a brief overview of the participating systems. As is customary in the CheckThat! Lab, we release all the datasets as well as the evaluation scripts to the research community. This will enable further research on identifying challenges with fact-checking numerical claims that can assist various stakeholders, such as fact-checkers, financial research analysts, and policymakers.
There has been growing interest in developing tools [
This task focuses on verifying claims with numerical quantities and temporal expressions. Numerical
claims are defined as those requiring validation of explicit or implicit quantitative or temporal details.
Participants must classify each claim as True, False, or Conflicting based on a short list of evidence.
Each claim is accompanied by the top-100 pieces of evidence retrieved using BM25 from our collection.
The collection was carefully curated through pooling evidences from retrieval using diferent query
understanding mechanisms [
The CheckThat! 2025 lab was held in the framework of CLEF 2025 [
T1: Should we check an opinion piece?
T2: Can we extract
claims and normalize them?
T4b: Can we identify mentioned scientific papers?
T3: Can we fact-check numerical claims? check-worthiness
estimation T4a: Can we retrieve scientific claims? verified claim
retrieval past verified claim supporting evidence retrieval
claim verification
TRUE FALSE OTHER full CheckThat! identification and verification pipeline, highlighting the four tasks targeted in this seventh edition of the lab: Task 1 on subjectivity, Task 2 on claims extraction & normalization, Task 3 on numerical claims (this paper), and Task 4 on scientific web discourse.
The remainder of the paper is organized as follows: Section 3 describes the datasets released with the task. We present the evaluation setup in Section 4. Section 5 discusses the system submissions and the oficial results. Section 2 presents some related work. Finally, we provide some concluding remarks in Section 6.
Automated claim-verification is key to mitigating growing misinformation [
To combat real-world misinformation, verifying claims containing numerical information is especially
important. Such claims, citing statistics, figures, or time spans are often perceived as more credible,
a phenomenon known as the numeric truth efect [
Among those that focus on simple statistical claims, [
QuanTemp [
The focus on claim verification has been a focus previous editions of the CLEF CheckThat!lab in
2018-2023 [
Following this tradition, we ofer a claim veracity prediction task but particularly focused on numerical claims across three languages: English, Spanish and Arabic.
Example: Claim decomposition example Claim: Discretionary spending has increased over 20-some percent in two years if you don’t include the stimulus. If you put in the stimulus, it’s over 80 percent. [Decomposition]: [Q1]: Has discretionary spending increased in the past two years? [Q2]: Does the increase in discretionary spending exclude the stimulus? [Q3]: Is there evidence to support the claim that Claim: A claim is designated as numeric if the numeric aspect is one of the crucial aspects to be verified in the claim. While there culd be other important aspects that may determine if a claim is correct or not, if numerical claim is one of the aspects it is still a numerical claim.
Examples: For instance in the claim Example 1: “A man wearing justice for breonna taylor shot and killed 3 men in a retired cops bar." Here there are several aspects to be verified: that the man was wearing the said t-shirt linking him to BLM protests, he killed 3 men and the act was carried out at retired cops bar.
While there are 3 aspects one crucial aspect is verification of fact that if he killed 3 men. because if this number say was misrepresented it would cause more panic due to spread of such misinformation. Example 2: “The chattsigarh police conveyed that naxal terrorists were involved in blast on January 22, 1794 " Here while the important aspect to be verified seems to be the terrorist group part, it is also a temporal claim as it is crucial to verify if blasts occured on said date.
The dataset contains multigenre content in Arabic, English, and Spanish. The dataset is collected from various fact-checking domains through Google Fact-check Explorer API2, complete with detailed metadata and an evidence corpus sourced from the web. Our pipeline filters out numerical claims for the task. An overview of dataset statistics is shown in Table 1. 2https://toolbox.google.com/factcheck/apis Extraction of quantitative segments: To extract numerical claims and filter out non-numerical ones, we employ the constituency parse of the original claim as shown in Figure 4. Figure 4 demonstrates how quantitative segments are identified using constituency parse of an English claim.
The nodes with the cardinal number POS tag “CD” are identified from the constituency parse. To avoid false positives (for example: “The one and only”), we then parse these nodes’ ancestors and extract noun phrases from their least common ancestors. Using these noun phrases as root nodes, we perform a prefix traversal of their subtrees. We then filter from set of all claims, those with at least one quantitative segment, as numerical claims for our dataset.
This approach still has one limitation, as it may include claims with non-quantitative terms like “Covid-19” or “F-35". To remedy this, we require more than one quantitative segment, excluding any nouns like “Covid-19” mentions, to qualify as a numerical claim. Our self-assessment of 1000 sample claims from English from the dataset indicates a 95% accuracy of our approach.
We follow the same process for Arabic. For extraction of Spanish numerical claims, the process is similar with exception of POS tag used for identifying numerical spans. The POS tag corresponds to “NUM" instead of “CD". We follow a similar manual evaluation process for Spanish and observe an accuracy of 97% demonstrating that our approach to identify quantitative segments and extract numerical claims is robust across languages.
Data characteristics: We use the train and validation sets from the English dataset released in [
English: For the English test set we collect new real-world English numerical claims additionally to
the evaluation set released in [
Arabic: The Arabic dataset only consists of claims belonging to the categories True and False for verification, as real-world distribution of conflicting claims for Arabic is too low. The claims in Arabic dataset collected from diverse online sources, including news outlets, social media, and professional fact-checking websites. Our goal was to curate a dataset that captures a wide range of topics such as politics, health, and economics, where numerical misinformation is common. We also incorporated verified claims from AraFacts [ 41], a large dataset of professionally fact-checked Arabic claims. A distribution of diferent categories of numerical claims along with the examples in as shown in Table 4.
Spanish: We employ a similar approach adopted for English to filter numerical claims with the exception of change in POS tag used. The guidelines shown in Figure 3 were employed by an expert in the language to verify the correctness of the curated numerical claims. A distribution of diferent categories of numerical claims along with the examples in as shown in Table 3.
Evidence Collection: Evidence for claims in all languages were obtained from search engines by excluding fact-checking websites to avoid leakage of fact-checker justification and verdict. For each claim, we decompose them to yes/no type sub-questions as shown in Figure 2, and issue the original claim and generated sub-questions as queries to the search engines. Additionally, for English, evidences are also obtained by other decomposition approaches like subclaim generation to increase diversity of evidence pool. All evidences are pooled to form the collection.
The numerical claim verification task primarily involved classifying the claim given the evidence pool to one of the three classes True, False or Conflicting by verifying aspects of the claim against evidence. While it was posed as a three-way classification task for English and Spanish, it was a two-way /True or False) classification task for Arabic. AKðPñ» ðQ ®Ë ©KQå PAJ.Jk@ Ë@
. ñJË Interval Comparison
We ranked the participants based on the Macro-F1 scores and not accuracy to account for class imbalance. Additionally, participants were also asked to report per-class F1 scores to measure if the systems performed reasonable well across diferent veracity classes.
Apart from the evidence collection mentioned in Section 3, we also provide the evidences from firststage retrieval. For each claim we retrieve top-100 evidences using BM25 based on Elasticsearch implementation. Top-100 evidences were independently provided for each decomposition approach to ofer flexibility for participants to use any decomposition approach.
Participants were encouraged to apply any re-ranking method to the provided evidence. They were also allowed to use their own retrieval approaches over the full document collection, as the BM25 results were ofered solely for convenience. For claim verification, participants were free to use any model of their choice.
Table 5 shows the performance of the oficial submissions on the test set. The oficial run was the last valid blind submission by each team. The table shows the runs ranked on the basis of the macro-F1 and includes all three languages.
A total of 258 valid runs were submitted by 13 unique teams across all languages. Among them, 10 teams participated in fact-checking English numerical claims, while 4 teams submitted runs for Spanish and Arabic. The final leaderboard in Table 5 includes 12 teams across languages, as one of the submissions was withdrawn from the leaderboard.
English: A total of 10 teams participated in fact-checking numerical claims in English. Most of the
teams employed BM25 evidence followed by re-ranking using cross-encoder. Several teams employed
creative approaches like data augmentation by translation from claims in Arabic and Spanish. At least 6
teams employ LLM based approach to reason over the evidence and provide claim veracity predictions.
Only one of the teams managed to outperform the baseline reported in [
Arabic: Four teams participated in Arabic with LIS being the top performing system with surprisingly high macro-F1 of 96.15 %. This system is the same as their system employed for English claims. Since they employ LLMs with ability to handle multiple languages, their solution generalizes beyond English. The high gains in Arabic can also be explained by the task not having a Conflicting task, and it’s reduction to binary classification of True or False compared to English and Spanish. The main advantage of system from LIS over other systems which underperform in Arabic seems to be improvement of retrieval using dense retrieval approaches like Linq-Embed-Mistral and fine-tuning LLMs instead of prompting for veracity prediction / NLI.
Spanish: We observe a trend in Spanish similar to that of English. Spanish proves to be much harder due to the three-way classification task, where conflicting is hard to detect due to multi-aspect nature of the claim and granularity of reasoning required to identify half-true aspects of the original claim. While LIS team outperforms other teams, they only manage to attain a macro-F1 of 50.34. This could also be explained by lack of MTEB evaluations in Spanish for Linq-Embed-Mistral. Hence, better alternatives to this retrieval model could exist. It also highlights the need for improving numerical reasoning and parsing capabilities of LLMs when employed for claim verification. As very few benchmarks exist for Spanish fact-checking and almost none for numerical fact-checking, the drop in performance could be explained by lack of suficient training data for fine-tuning the LLM employed for the veracity prediction part.
Among all participating teams, LIS was the top performer across all languages. TIFIN, NGU_Research, DS@GT-CheckThat! performed well in the respective languages they participated. Most teams employed generative models like gpt-4o-mini or Qwen LLMs to decompose claims, followed by BM25 based retrieval for retrieving evidence and transformer based cross-encoder models for re-ranking the evidences. For claim verification, fine-tuned transformer based NLI models were employed by some teams where transformers were trained as discriminative models on the training sets provided. Some teams employed prompting based approaches to leverage LLMs like gpt-4o-mini or reasoning models like deepseek-r1 to perform claim verification.
Team LIS [42] used QwQ-32B to generate question followed by Linq-Embed-Mistral to retrieve
evidence from the corpus by combining the questions and claims. Mistral-Small-24B-Instruct-2501 was
ifne-tuned to obtain the final veracity labels. The Qwen model seem to overcome certain limitations
associated with GPT-3.5 and GPT-4 series models used in baselines [
Team DS@GT-CheckThat! [43] performed pre-processing to normalize the number and dates of the claims and decomposed questions from these claims. They employed GPT-4o-mini to decompose the claims. BM25 was employed for first stage retrieval to prioritize documents relevant to the claim and sub-questions. This is followed by re-ranking the documents using cross-encoder/ms-marcoMiniLM-L-12-v2 or mixedbread-ai/mxbai-rerank-large-v1. The main workhorse model for the veracity classification was ModernBERT - an optimized model based on the BERT architecture, that can natively support longer sequence length.
Team TIFIN [44] employed inverse class weighting in the claim verification step to address class imbalance and give greater importance to minority classes. They also used strategies such as oversampling to balance training examples and label smoothing to prevent the model from becoming overconfident in its predictions. Additionally, they incorporated Focal Loss to fine-tune the verification model microsoft/deberta-large-mnli using LoRA, allowing the model to focus on harder examples. To further enhance performance, they used the ibm-granite/granite-3.3-8b-instruct model to summarize contexts before feeding them to the verification model. An interesting insight presented by the authors is that performance does not scale linearly with model size – smaller LLMs outperformed 70B-scale LLaMA models, challenging assumptions based on scaling laws. Their approach also demonstrated the benefits of multilingual data augmentation in improving claim verification performance.
Team NGU_Research [47] employed a hybrid retrieval approach, experimenting with various pretrained encoder-based models – including BGE, E5, and Gemini – as embedding models. They ultimately selected pretrained embeddings from OpenAI’s text-embedding-3-large model, combined with BM25 filtering using Qdrant database collections for each language. For claim verification, they used DeepSeek and GPT-4o-mini on the retrieved evidence.
Team ClaimIQ [45] presented their core approach of fine-tuning LLMs using Low-Rank Adaptation (LoRA) for the task of claim verification, combined with existing retrieval and ranking strategies to procure evidence. The authors observed that this approach outperformed prompting-based methods and other NLI models on the validation set. However, these performance gains did not generalize to the evaluation set, highlighting that fine-tuning larger LLMs can lead to overfitting. This further underscores the challenging nature of the task, which requires models not only to classify but also to perform numerical contextualization and reasoning.
Team Fraunhofer_SIT [46] follows a three-stage architecture: (1) evidence candidate retrieval using dense vectors, (2) re-ranking using a fine-tuned cross-encoder, and (3) final claim classification using a large NLI model (Roberta-large-MNLI). The authors first pre-compute document embeddings using sentence-transformers/all-MiniLM-L6-v2 and store them in a FAISS index to enable eficient nearest-neighbor retrieval. At inference time, the top-100 candidate evidence snippets are retrieved for each claim. These are then re-ranked using a custom cross-encoder based on cross-encoder/ms-marcoMiniLM-L-6-v2, fine-tuned in a contrastive learning setup detailed as follows.
To improve re-ranking, the authors opted not to use the vanilla MS MARCO model employed in the baseline. Instead, they leveraged weak supervision by using gold-labeled evidence snippets for training and validation claims. These gold snippets were summarized using LLaMA 3.1-8B to remove irrelevant or noisy content, resulting in cleaner positive examples. The cross-encoder was fine-tuned using a contrastive approach, where the claim served as the anchor, the summarized gold evidence as the positive, and the top-100 BM25-retrieved documents as negatives. Although they ranked fifth on the leaderboard, their approach demonstrates that enhancing retrieval can significantly boost downstream NLI performance without relying on expensive LLM-based methods for claim verification.
The KSU team employed BM25 evidence with cross-encoder based re-ranking. Additionally, since evidence snippets are long, they employed a unsloth/Qwen3-8B-bnb-4bit model to filter out most important snippets. This was followed by a fine-tuned NLI model over the snippets for veracity prediction.
However, even the top performing system from LIS did not reach the best possible performance
attainable using the provided evidence collection, as outlined in [
We presented an overview of Task 3 of the CLEF 2025 CheckThat!lab, which focused on fact-checking
numerical claims. Approaches ranged from prompting Large Language Models (LLMs) to fine-tuning
open-source LLMs of smaller parameter scales. Some participants also focused on improving evidence
re-ranking and demonstrated this can improve claim verification performance with smaller transformers
models. Some participants also employed creative data enrichment techniques by translating claims
from other languages to train the NLI model on a larger, well-balanced augmented dataset. While
some efort was made on improving loss functions used to train the NLI model this did not yield any
statistically significant improvements. Only the top-1 result in English leaderboard outperformed the
strong baseline reported by organizers in [
During the preparation of this work, the authors used Grammarly in order to: Grammar and spelling check and rewording some text. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the publication’s content. No other generative AI or model was used in preparation of this paper.
The work of F. Alam and M. Hasanain is partially supported by NPRP 14C-0916-210015 from the Qatar National Research Fund, part of Qatar Research Development and Innovation Council (QRDI). 2018, Avignon, France, September 10-14, 2018, Proceedings, 2018, pp. 372–387. doi:10.1007/ 978-3-319-98932-7_32. [40] P. Nakov, A. Barrón-Cedeño, G. Da San Martino, F. Alam, J. M. Struß, T. Mandl, R. Míguez, T. Caselli, M. Kutlu, W. Zaghouani, C. Li, S. Shaar, G. K. Shahi, H. Mubarak, A. Nikolov, N. Babulkov, Y. S. Kartal, J. Beltrán, The clef-2022 checkthat! lab on fighting the covid-19 infodemic and fake news detection, in: M. Hagen, S. Verberne, C. Macdonald, C. Seifert, K. Balog, K. Nørvåg, V. Setty (Eds.), Advances in Information Retrieval, Springer International Publishing, Cham, 2022, pp. 416–428. [41] Z. Sheikh Ali, W. Mansour, T. Elsayed, A. Al-Ali, AraFacts: the first large arabic dataset of naturally occurring claims, in: Proceedings of the Sixth Arabic Natural Language Processing Workshop, 2021, pp. 231–236. [42] Q. T. Le, I. Badache, A. Yacoub, M. E.-A. Hamri, Lis at checkthat! 2025: Multi-stage open-source large language models for fact-checking numerical claims, in: [51], 2025. [43] M. Heil, A. Pramov, Ds@gt at checkthat! 2025: Evaluating context and tokenization strategies for numerical fact verification, in: [51], 2025. [44] B. Hazarika, P. Devadiga, P. K. Rajpoot, A. U. Baliga, K. Gurumurthy, M. Jain, M. Sharma, A. Shrivastava, A. Suneesh, A. B. Suresh, Tifin at checkthat! 2025: X-verify - multi-lingual nli-based fact checking with condensed evidence, in: [51], 2025. [45] A. S. Anik, M. F. K. Chowdhury, A. Wyckof, S. R. Choudhury, ClaimIQ at CheckThat! 2025: Comparing prompted and fine-tuned language models for verifying numerical claims, in: [ 51], 2025. [46] A. Runewicz, P. M. Ranly, I. Vogel, M. Steinebach, Fraunhofer SIT at CheckThat! 2025: Multiinstance evidence pooling for numerical claim verification, in: [51], 2025. [47] M. A. Abdallah, R. M. Fekry, S. R. El-Beltagy, NGU_Research at checkthat! 2025: An LLM based hybrid fact-checking pipeline for numerical claims, in: [51], 2025. [48] L. Duesterwald, A. Arora, C. Cardie, CornellNLP at CheckThat! 2025:hybrid llama–gpt-4 ensembles with confidence filtering for numerical claim verification, in: [51], 2025. [49] M. d. C. Toapanta Bernabé, M. A. García Cumbreras, L. A. Ureña López, D. Mora, UGPLN at CheckThat! 2025: Meta-ensemble transformers for numerical claim verification in spanish, in: [51], 2025. [50] G. Acosta, E. Morales, H. Gómez-Adorno, UCOM_UNAM_PLN @ Checkthat 2025: Evaluating llms in a two-step architecture for numerical fact checking, in: [51], 2025. [51] G. Faggioli, N. Ferro, P. Rosso, D. Spina (Eds.), Working Notes of CLEF 2025 - Conference and Labs of the Evaluation Forum, CLEF 2025, Madrid, Spain, 2025.