In this paper, we propose an integrated approach for medical image retrieval. In particular, we present a series of experiments in medical image retrieval task. There are three main goals for our participation of this task. First, we will test traditional well-known weighting models used in text retrieval domain, such as BM25, TFIDF and Language Model (LM), for context-based image retrieval. Second, we will evaluate statistical-based feedback models and ontology-based feedback models. Third, we will investigate how content-based image retrieval can be integrated with these two basic technologies of traditional text retrieval. The experimental results have shown that 1) traditional weighting models can work well in context-based medical image retrieval task especially when the parameters are tuned properly; 2) statistical-based feedback models can improve the retrieval performance when a small number of documents are used; however, the medical image retrieval can not benefit from ontology-based query expansion; 3) the retrieval performance can be slightly boosted by integrating content features.
For the first year of the participation in ImageCLEFmed task, we first test traditional wellknown weighting models used in text retrieval domain, such as BM25, TFIDF and LM, for contextbased image retrieval. Since the text context information (captions) is very short, which is different from traditional adhoc text collections, it is necessary to test and adapt traditional weighting models for this particular task. Second, on the basis of the baseline results, we use statisticalbased pseudo relevance feedback and ontology-based (MeSH 1) query expansion approaches to enhance the retrieval performance. Finally, we note that many images share the same context text for comparison reason in an article, but the circumstance is always that only one of these images is what we are looking for. So it is impossible for us to filter other images with the same context text using only the context-based image retrieval technologies. We explore different image content features to enhance context-based image retrieval technologies.
The remainder of this paper is organized as follows. In section 2, we describe the comparisons of basic retrieval models. In section 3, we present the experimental results for statistical-based and ontology-based query expansion. In section 4, we propose an integrated approach for medical image retrieval. In section 5, we conclude the paper with a discussion of our findings and a look at future work. 2
In the previous medical image retrieval tasks, a number of different information retrieval (IR) toolkits, such as Lemur, Jirs and Lucene, are used as the basic retrieval systems for context-based medical image retrieval. However, there is no systematic comparison of different weighting models for ImageCLEFmed task. In addition, it is not clear that whether the default parameters in these models empirically tuned for traditional adhoc datasets are optimal.
In this paper, we have made comparisons for four well-known weighting models: BM25 [
BM25 JM - LM TFIDF DFR ! = k1 ((1 (k1 + 1) tf b) + b dl=avdl) + tf log
N
n + 0:5 n + 0:5 (k3 + 1 ) qtf
k3 + qtf ! = (1 + 1 tf
F reqT otColl l Ft
) ! = qtf
k1 tf tf + k1 (1 b + b dl ) avdl log(1 +
N n
) ! = T F qtf
N ORM
N + 1 loge( n exp )
T F = tf log2(1 + avdl=dl) N ORM = (tf + 1)=(df (T F + 1)) n exp = idf (1
e f ) f = qtf =df (1) (2) (3) (4) where w is the weight of a query term, N is the number of indexed documents in the collection, n is the number of documents containing the term, R is the number of documents known to be relevant to a specific topic, r is the number of relevant documents containing the term, tf is within-document term frequency, qtf is within-query term frequency, dl is the length of the document, avdl is the average document length, nq is the number of query terms, the kis are tuning constants (which depend on the database and possibly on the nature of the queries and are empirically determined), K equals to k1 ((1 b) + b dl=avdl).
In our experiments, the values of k1, k2, k3 and b in the BM25 function are set to be 1.2, 0, 8 and 0.75 respectively; the value of is set to 0.15. In addition, the image context texts are preprocessed in the same way for all experiments in order to make reasonable comparison. We use blank delimiter to separate words for indexing and searching and stopwords are removed. Beside these two simple steps, no further technologies have been used.
The pseudo relevance feedback method used in our experiments is DFR-based weighting model
described in [
The main goal of this set of experiments is to investigate how many top documents and terms should be used for query expansion. For the limitation of space, we only present experimental results on the basis of BM25 model.
Form table 5 we can see, in general, the performance can be boosted if we can set the parameters properly. When the number of documents for query expansion increases from 5 to 10, the performance drops quickly. The results in table 5 suggests that only a very small number documents are useful for query expansion in context-based medical image retrieval task. 3.2
In medical domain, terms are highly synonymous and ambiguous. This motivates us to investigate using ontology to expand the original query terms.
The Medical Subject Headings (MeSH) is a thesaurus developed by the National Library of Medicine. MeSH contains two organization files, an alphabetic list with bags of synonymous and related terms, and a hierarchical organization of descriptors associated to the terms. A term is composed by one or more words.
We have used the longest match approach to recognize the MeSH terms in a query. In particular, if all the words of a term are in the query, we add our synonymous terms to the query. To compare the words of a particular term and those of the query, we rst put all the words in lowercase and we do not remove stop words. In order to reduce the number of terms that could expand the query, only three categories of MeSH terms (A: Anatomy, C: Diseases, E: Analytical, Diagnostic and Therapeutic Techniques and Equipment) have been used for query expansion. Table 3.2 presents the MeSH-based query expansion results under four different models.
Unfortunately, MeSH-based query expansion approach does not work well as we expected. Our conjecture is that MeSH-based query expansion may also bring negative terms into query, especially the abbreviation terms. In addition, when the JM-LM is used as the basic retrieval model, the performance drops remarkably. This is another evidence that the performance of JM-LM is not steady. 4
Content-based Image Retrieval (CBIR) systems enable users to search a large image database by issuing an image sample, in which the actual contents of the image will be analyzed. The contents of an image refer to its features – colors, shapes, textures, or any other information that can be derived from the image itself. This kind of technology sounds interesting and promising. The key issue in CBIR is to extract representative features to describe an image. However, this is a very very difficult research topic.
According to ImageCLEFmed conference notes [
1. Color and Edge Directivity Descriptor (CEDD): is a low level feature which
incorporates color and texture information in a histogram [
The final rank list is attained by merging the context-based retrieved score (Scontext) and content-based similarity score (Scontent). In particular, we use linear combination. The formula is described as follows.
score = (1 ) Scontext +
Scontent (6)
From table 7, we can see that retrieval performance can be slightly boosted by integrating content features. Among these three features, CEDD can improve the performance most. However, more representative features are needed to be developed. In this study, we first evaluate four well-known weighting models for context based medical image retrieval. The performances of the four weighting models are comparable, but DFR weighting model works best under default settings. JM-LM model is not steady for this task, but if the parameter can be tuned properly, it is still promising. Second, we investigate query expansion technologies for this task. In general, statistical-based QE method outperforms ontology-based methods. The experimental results also suggests that only a small number of top ranked documents are useful for statistical-based QE method. Ontology-based methods sound interesting and useful, however the actual performance is not good. More sophisticated processing for this kind of methods is needed. Finally, we explore three content features for content-based medical image retrieval. The experimental results have shown that retrieval performance can only be slightly improved. The current features extracted from images may not be representative enough to capture the characteristics of images. Better features are required to improve CBIR.
In the future work, we will work on the following two directions. First, we will use data-driven approaches to choose optimal parameters for statistical-based QE. Second, we will explore the correlation of different content features of images. In addition, if more features can be integrated into medical image retrieval properly, we believe the retrieval performance can be further improved. 6
This research is jointly supported by NSERC of Canada, the Early Researcher/Premier’s Research Excellence Award, Natural Science Foundation of China (No. 60373095 and 60673039) and the National High Tech Research and Development Plan of China (2006AA01Z151).