UB at CLEF 2005: Medical Image Retrieval Task Miguel E. Ruiz and Silvia B. Southwick State University of New York at Buffalo School of Informatics Department of Library and Information Studies 534 Baldy Hall, Buffalo NY, USA Email: meruiz@buffalo.edu Abstract: This work was part of SUNY at Buffalo’s overall participation in cross-language retrieval of image collections (ImageCLEF). Our main goal was to explore the combination of Content-Based Image Retrieval (CBIR) and text retrieval of medical images that have clinical annotations in English, French and German. We used a system that combined the content-based image retrieval system GIFT and the well-known SMART system for text retrieval. Translations of English topics to French were performed by mapping the English text to UMLS concepts using the 2005 UMLS meta-thesaurus. Results show that combining both CBIR and Text retrieval yields significant improvements of retrieval performance. System Description For this year’s participation we used a system similar to the one we designed for Image-CLEF 2004 (Ruiz & Srikanth, 2005). This system combines the GNU Image Finding Tool (GIFT) and the SMART information retrieval system(Salton, 1971). This year the queries included a short text description as well as one or more sample images. The image retrieval was performed using the ranked list of retrieved images generated by GIFT which was made available to all participants of the image retrieval track. The SMART system was used to create a multilingual database with all the case descriptions of the 4 collections of images (Casimage, MIR, Pathopic, and PIER). The English text associated with each query was processed using MetaMap to obtain a list of UMLS concepts (Aronson, 2001). MetaMap uses an NLP approach to map UMLS concepts to free text. The input text is passed though a series of processes that includes identification of paragraph, sentences, phrases and tokens; generation of variants of the resulting phrases, and the corresponding mapping of these phrases to UMLS concepts. MetaMap uses three models that are related to how accurate should be the matching of the text against the UMLS concepts (strict, moderate or relaxed). The UMLS concepts were identified using the strict model. For these concepts we locate the corresponding French terms associated and use these terms as translation of the English query. The French terms were added to the original English query to generate a bilingual English-French query that was submitted to SMART. Figure 1 shows a schematic representation of the system used in this task. Note that although this is a simple approach there are some limitations. The major problem that we face is that not all English UMLS concepts have French translations. This coverage problem might cause that some queries might not have any translations or that some important query terms might not be translated. The current number of concepts with French translations is 24,721 out of more that 500K UMLS concepts. However, when we reviewed our queries all of them had at least one concepts with French translation. This indicates that the small French translations are probably the most commonly used concepts. Figure 1. System for Image-CLEF 2005 The organizers of the conference provided a list of the top 1000 retrieved images returned by GIFT for each of the 25 topics. We used this list as the output of the CBIR system. From the textual part of the query we used only the English version provided in the query. This Text was processed with MetaMap,. The current Version of UMLS includes vocabularies with translations in 13 languages (Including French and German which are the ones of interest for this task). For these experiments we decided to use the French terms present in UMLS as translations. We decided not to use German due to the fact that the German documents come from the Pathopic collection and constitute a parallel collection of English and German. Since our initial query is in English we consider redundant to try to get German translations for the topics. However, German terms might be added if they rank high in the automatic relevance feedback process. The corresponding English and French UMLS terms associated with the concepts identified by MetaMap were added to the original English Queries. The text was processed by SMART and the top 10 cases retrieved (together with the cases from top 10 image retrieved by GIFT) are used for expand the query. Observe that this is a multilingual expansion since terms in French, German or English could be added by the query expansion process. The images associated with each case are given the same retrieval score from the text retrieval. Finally we combined the image scores obtained from the CBIR system and from the text retrieval system. We weighted this combination and the results indicate that a 3:1 weighting in favor of text retrieval achieve the best performance in our system. Results We submitted five runs that included different weights for the contribution of text and images, and several variations on the number of terms used for expansion. The best combination of our official results was obtained by weighting the text results 3 times higher than the visual results. The parameters for the relevance feedback used the top 10 results (both images and text cases), and expand the queries with the top 50 ranked terms using Rocchio’s formula (with α=8, β=64, γ=16). This combination showed a significant improvement for retrieval performance (35% with respect to using text only and 150% using only image retrieval). Our runs were ranked 4th overall and our system was among the best 3 participating in the conference. Table 1. Summary of results Parameters MAP Text only n=10, m=50 0.1746 ( UBimed_en-fr.T.Bl ) Visual only GIFT results 0.0864 Text and visual UBimed_en-fr.TI.1 a= 1, b=3 0.2358 n= 10, m=50 UBimed_en-fr.TI.2 a= 1, b=1 0.1663 n= 10, m=50 UBimed_en-fr.TI.3 a= 2, b=5 0.2195 n= 10, m=150 UBimed_en-fr.TI.4 a= 2, b=5 0.1742 n= 10, m=50 UBimed_en-fr.TI.5 a= 1, b=3 0.1784 n= 10, m=20 Our preliminary analysis showed significant improvements of 35% over text-only and 173% over queries that used only visual features. We still need to do the query by query analysis to try to identify if there are specific types of queries for which our approach works better. Our future research plans include exploring in more detail the contribution of the translation using MetaMap and UMLS, types of queries that benefit the most from either text or image retrieval, and sensitivity of the system to different parameters. Acknowledgements We want to thank the National Library of Medicine for their support of Miguel Ruiz as a visiting faculty at the National Library of Medicine and for allowing us to use UMLS and MetaMap for this research. We want to thank AT&T for supporting this research by providing the necessary hardware used in our experiments through a small grant for research at the UB School of Informatics. Bibliography Aronson, A. (2001). Effective Mapping of Biomedical Text to the UMLS Metathesaurus: The MetaMap Program. Paper presented at the AMIA. Ruiz, M. E., & Srikanth, M. (2005). UB at CLEF2004 Cross Language Medical Image Retrieval. Paper presented at the Fifth Workshop of the Cross-Language Evaluation Forum (CLEF 2004), Bath, England. Salton, G. (1971). The SMART Retrieval System: Experiments in Automatic Document Processing. Englewood Cliff, NJ: Prentice Hall.