This paper describes the participation of MIRACLE research consortium at the ImageCLEFmed task of ImageCLEF 2008. The main goal of our participation this year is to compare among different topic expansion approaches: methods based on linguistic information such as thesauri or knowledge bases, and statistical techniques based on term frequency. Thus we focused on runs using text features only. First a common baseline algorithm was used in all experiments to process the document collection: text extraction, medical-vocabulary recognition, tokenization, conversion to lowercase, filtering, stemming and indexing and retrieval. Then this baseline algorithm is combined with different expansion techniques. For the semantic expansion, the MeSH concept hierarchy using UMLS entities as basic root elements was used. The statistical method consisted of expanding the topics using the apriori algorithm. Relevance-feedback techniques were also used.
The architecture of our system is composed of four different modules: the textual (text-based) retrieval module, which indexes medical case descriptions in order to search and find the most relevant ones to the text of the topic; the expander module, which performs the expansion of the content of documents and/or topics with related terms using textual or statistical algorithms; the relevance-feedback module, which allows to execute reformulated queries that include the results of an initial seed query; and, finally, the result combination module, which uses OR operator to combine, if necessary, the result lists provided by the previous subsystems. Figure 1 gives an overview of the system architecture.
Instead of using raw terms, the textual information of both topics and documents is parsed and tagged to unify all
terms into concepts of medical entities. This is similar to a stemming or a lemma extraction process, but the
output, instead of the stem or lemma, is the medical entity to which the term relates. The result is that concept
identifiers [
Text Extraction: Ad-hoc scripts are run on the files that contain information about the medical cases so as to extract the annotations and metadata enclosed between XML tags.
Medical-vocabulary Recognition: All case descriptions and topics are parsed and tagged using the UMLS-based terminological dictionary to identify and disambiguate medical terms.
Tokenization: This process extracts basic textual components, detecting and isolating punctuation symbols. Some basic entities are also detected, such as numbers, initials, abbreviations, and years. So far, compounds, proper nouns, acronyms or other types of entity are not specifically considered. The outcomes of this process are only single words, years in numbers (e.g. 1995, 2004, etc.) and tagged entities. 4.
Conversion to lowercase: All terms are normalized by changing all uppercase letters to lowercase.
5. Filtering: All words recognized as stopwords are filtered out. Stopwords in the target languages were
initially obtained from the University of Neuchatel’s resources page [
Porter stemmers [
This common baseline algorithm is complemented and combined with different expansion techniques in order to
compare the improvement given by semantic- versus statistical-based techniques. For the semantic expansion,
we used the MeSH concept hierarchy [
The statistical method consisted of expanding the topics using the Agrawal’s apriori algorithm [
Finally, relevance-feedback techniques were also used. The top M UMLS entities of each of the top N result documents were extracted and weighted by a factor that is proportional to their document frequency to reformulate a new query that is executed once again to get the final result list.
Experiments are defined by the choice of different combinations of the previous modules with the different topic expansion techniques, and including relevance-feedback or not. Table 2 shows the complete list of submitted runs.
Method stem + stopwords + tagged with UMLS thesaurus baseline + Apriori topic expansion baseline + MeSH topic expansion baseline + Relevance-Feedback (N=5, M=5) baseline + Relevance-Feedback (N=10, M=5) baseline + MeSH topic expansion + Relevance-Feedback (N=10, M=5) baseline + MeSH topic expansion + Relevance-Feedback (N=10, M=5) baseline + MeSH topic expansion + Relevance-Feedback (N=10, M=5) Results are presented in the following table, which shows the run identifier, the number of relevant documents retrieved, the mean average precision (MAP), and the precision at 5, 10, 30 and 100 first results. The best results are highlighted in bold.
A preliminary analysis of the results, given the low precision values obtained in the experiments that make use of the relevance-feedback methods, shows that the reranking algorithm used for combining the different result lists is likely to be the main reason for the disappointing results. However, this impression has to be confirmed with a more in-depth analysis. Another probable cause is the choice of the OR operator to combine the terms in the topic to build up the query. Due to time constraints to prepare this report, we were unable to repeat our experiments with the AND operator, but we think that MAP values should be significantly higher using this operator.
In addition, experiments using French and German languages get a very low precision. A possible explanation is that the process of entity unification (detection) for those languages is poor, due to the reduced coverage of the knowledge base. We will try to complete and expand the thesaurus for those languages with other available resources.
This work has been partially supported by the Spanish R+D National Plan, by means of the project BRAVO (Multilingual and Multimodal Answers Advanced Search – Information Retrieval), TIN2007-67407-C03-03 and by Madrid R+D Regional Plan, by means of the project MAVIR (Enhancing the Access and the Visibility of Networked Multilingual Information for the Community of Madrid), S-0505/TIC/000267.