Gastrointestinal (GI) image analysis is critical for early diagnosis and treatment of GI diseases, which remain a leading cause of global morbidity and mortality. Visual Question Answering (VQA) in medical imaging enables interpretable and interactive AI systems to support clinical decision-making. This paper presents our submission to the ImageCLEFmed 2025 MedVQA task, which targets medical VQA on gastrointestinal endoscopic images using the Kvasir-VQA dataset. We evaluate two primary approaches: (1) a multimodal Chain-of-Thought (CoT) reasoning framework that decomposes questions into interpretable reasoning steps, and (2) a simple fine-tuning strategy on large-scale generative models. Extensive experiments across multiple vision-language models, including Qwen2-VL and BLIP2, show that fine-tuning significantly outperforms CoT in both validation and test settings. Our best-performing model, achieves a METEOR score of 50 on the test set. We also carried out qualitative and quantitative analysis to demonstrate the strengths and weaknesses of our best performing approach, and hence suggesting some insights to tackle the most challenging aspects in the Kvasir-vqa task.
MedVQA has gained significant attention as a critical task in biomedical AI, requiring models to
generate accurate textual answers conditioned on visual medical images. Early MedVQA Approaches
addressed tasks with limited annotated data. For example, [
The development of robust and clinically relevant Visual Question Answering (VQA) systems for
medicine is heavily dependent on high-quality annotated datasets. Over the past few years, several
notable datasets have emerged, each addressing unique aspects of medical image understanding through
natural language queries. VQA-RAD [
The Medical Visual Question Answering (MedVQA) task aims to develop models that can accurately answer clinically relevant questions about gastrointestinal (GI) endoscopic images. Leveraging the Kvasir-VQA dataset, the task combines computer vision and natural language understanding to simulate expert-level diagnostic reasoning. Formally, given an input medical image and a natural language question associated with the image, the objective is to map the image-question pair to a natural language answer that is accurate and contextually grounded in the image.
In this paper, we propose exploring two approaches to tackle the ImageCLEFmed 2025 MedVQA (task 1). We first investigate how a multimodal chain of thoughts (CoT) system would perform on the Kvasir-vqa dataset. Then, we evaluate a simple finetuning strategy using the kvasir-vqa training dataset and other medical training data.
Chain-of-Thought (CoT) reasoning enables Large Language Models (LLMs) to explicitly decompose
complex questions into intermediate reasoning steps [
You are a Vision Language Model assistant which helps an experienced doctor interpreting accurately interpreting and answering clinical questions based on gastrointestinal images. Given the image, provide a reasonable rationale for the question: {QUESTION}. Please proceed with a step-by-step analysis and provide a rationale.
Subsequently, R is used to generate the final answer A, such that: A = LVLM(q, i, R). We relied on the following prompt:
You are a Vision Language Model assistant which helps an experienced doctor interpreting accurately interpreting and answering clinical questions based on gastrointestinal images. Given the image and the rationale: {RATIONALE}, Answer briefly the question: {QUESTION} with no extra text, rationales or explanation.
Since the generated rationale can be inefective with regard to the ground truth answer * , we trained the LVLM on answering Kvasir-vqa questions in order to reduce discrepancies between rationales and the actual answers. The LVLM is trained on the following cross-entropy loss:
ℒgen = − ∑︁ log (* , , , )
=1 where * is the ground truth answer, is the question, is the image, and is the rationale.
In the second approach, we performed answer generation using a generative model denoted G(·) with parameters Φ . Given a question and the associated image , the answer generator G is trained on the following cross-entropy loss over a batch of question-image pairs: (1) (2) ℒG = − ∑︁ log Φ(* | , ) =1
Where is the -th question, is the image associated with , * is the ground truth answer string for ( , ), Φ(* | , ) is the probability of generating the correct answer from the text-image pair, Φ are the parameters of the multimodal answer generator.
We implemented all our experiments in Pytorch [
We evaluated both Chain-of-Thought (CoT) and fine-tuned (FT) models using BLEU, ROUGE, and METEOR scores. The Qwen2-VL and BLIP2 model architectures were evaluated for both methodologies. We additionally assessed our best performing model using the Exact Match metric to perform qualitative and quantitative analysis. Where in the image is center; center-left; the abnormality? center-right
Table 2 show that the BLIP2 model fine-tuned on the Kvasir-VQA dataset achieved the best overall performance on the Kvsair-vqa validation set. Note that, to achieve the best performance on the test set, What color is the red; pink; white anatomical landmark? If more than one separate with ; What is the size of the 5–10 mm polyp? Are there any abnor- ulcerative colitis malities in the image? Check all that are present.
What type of pro- gastroscopy
cedure is the image
taken from?
What type of pro- gastroscopy
cedure is the image
taken from?
we further finetuned the BLIP2 model on the training sets of PathVQA [
Furthermore, the performance gap between BLIP2-Flan-T5-XL and Qwen2-VL models is worth noting. Indeed, BLIP2-Flan-T5-XL consistently outperforms Qwen2-VL-7B-instruct whatever the training method and has comparable performance with Qwen2-VL-32B-instruct in the CoT setting despite their diference in model size. Besides, given that our experiment LoRA configuration reduces the number of trainable parameters, we found that: BLIP2-Flan-T5-XL has 4.7M, Qwen2-VL-7B has 2.5M and Qwen2VL-32 has 8.3M trainable parameters. This shows that BLIP2-Flan-T5-XL shows superior capabilities on the Kvasir-vqa task despite its relative size. We believe that encoder-decoder architectures such as BLIP2 are more suitable for VQA tasks as they allow to encode rich image features before generating the textual output, facilitating better multimodal alignment, while decoder-only models like Qwen2-VL must process the image and question together through a single stream, which may limit fine-grained control over visual and textual token interactions during generation.
Table 5 shows the exact match evaluation results of our best performing model (BLIP2-FT) by image category. We achieved the highest EM scores of 99.02% and 97.83% respectively for Normal Colon and Normal Esophagus image catgories. This due to the low variability in answers, as all the questions related to these image categories cover only yes/no question types which seem to be an easy task for BLIP2 model finetuned on similar data. We see in Table 4 the only examples of these image categories where our model wrongly predicts the yes/no questions. Moreover, Table 6 and Table 7 show that our model achieves respectively 96.23% and 91.94% EM scores on questions with yes/no answers whatever the image category.
Our best BLIP2 model also achieved solid EM performance on the following image categories: Cecum with 84.16%, Pylorus with 81.82%, and Esophagitis 79.22%, indicating its relative ability in identifying specific anatomical regions and whether some pathological signs are present. In contrast, our model struggled the most on questions related to the Polyp image category with the lowest EM score of 48.74%. On the one hand, answering questions about polyp requires the model to consistently identify more subtle image features and on the other hand, the polyp image category in the dataset cover a wider range of question types including among others: yes/no, color-related, counting and location-related questions. Similarly, the Instruments image category is also challenging for our model which yielded an EM score of 60.87%, as it also covers several question types which require distinguishing medical instruments from the background. These results suggest that the model may greatly benefit from more advanced and specific reasoning abilities such as visual spatial reasoning in order for example to accurately answer location-related questions for which our model achieves poor results (see Tables 6 and 7).
This paper presented two simple approaches for tackling the ImageCLEFmed 2025 MedVQA challenge using the Kvasir-VQA dataset. While the Chain-of-Thought approach ofered insights into the reasoning process behind answer generation, fine-tuning large generative models achieved significantly better performance across all evaluation metrics. Our experiments demonstrate the efectiveness of large vision-language models like BLIP2 when adapted to domain-specific medical tasks. Qualitative and quantitative analysis show that endowing the model with more complex visual reasoning abilities might improve the VQA performance on the questions related to the most challenging image categories namely Polyp and Instruments.
During the preparation of this work, we acknowledge the use of generative AI tools (Chat-GPT-4) for only spell checking, paraphrasing, and latex formatting purposes. After using Chat-GPT-4, we systematically reviewed and edited all the content as needed and take full responsibility for the publication’s content.
The following tables shows our best performing BLIP2 model results by answer ( or question type) on the validation split. 4 2 1 1 5 3 1 2 6 1 18 3 1 1 1 1