This paper presents our participation in ImageCLEF2011, in the two tasks: ad-hoc image-based retrieval and case-based retrieval, of the medical retrieval track. We participated through a simple IR model based on three hypotheses: 1) the amount of overlap between a document and a query, 2) the descriptive power of an indexing element, and 3) the discriminative power of an indexing element. We used three types of indexing elements: ngrams,
Each information retrieval (IR) model has its advantages and drawbacks. In
other words, an IR model may perform well in some cases but badly in others.
In general, there is no IR model performs well in all cases [
Our model is not an exception. Therefore, a general mechanism to evaluate the performance of an IR model and compare it with the performance of others is needed. In this context, there are many campaigns of evaluation in IR eld, e.g. CLEF1.
First, our model is a text-based retrieval and very simple model. It uses multiple types of indexing elements, e.g. ngrams, keywords, or concepts. The goals of this research are: 1) studying the performance of the model itself, using one of the indexing element types, 2) studying the e ects of using multiple indexing element types at the same time on the performance.
Second, CLEF (Cross-Language Evaluation Forum) is a yearly evaluation
campaign in Multilanguage information retrieval eld since 2000. ImageCLEF2
is a part of CLEF. It concerns searching medical images through documents that
contain text and images [
This year, ImageCLEF20113 [
Third, we participated in the last two tasks: ad-hoc image-based retrieval and case-based retrieval.
The test collection of this year 2011 contains, according to each task: 1) adhoc image-based retrieval: about 230,000 images with their text caption written in English and 30 queries written in three languages: English, French, and German. 2) case-based retrieval: about 55,000 articles written in English and 10 queries written in English.
This paper is structured as follows: Section 2 describes in details our model and the di erent types of indexing elements that are used. Section 3 presents some technical aspects of applying this model for ImageCLEF2011 test collection. Section 4 discusses the obtained results. We conclude in section 5. 2 2.1
Any IR model should contain two main components: indexing function and matching function. The goal of the indexing function is to convert documents and queries from their original form to another easy to use form.
Index : D [ Q ! E Where D set of documents Q set of queries E set of indexing elements E the set of all subsets of E
Concerning indexing elements, three di erent types are used: ngrams (N G), keywords (K), and concepts5 (C). Therefore, three indexing functions are exist (one for each type).
3 http://www.imageclef.org/2011
4 http://www.ncbi.nlm.nih.gov/pubmed/
5 "Concepts" can be de ned as "Human understandable unique abstract notions
independent from any direct material support, independent from any language or
information representation, and used to organize perception and knowledge" [
(4) Where ENG set of ngrams EK set of keywords EC set of concepts
We believe that no single type of indexing elements could completely
represent the content of documents and queries, because: 1) there is no perfect
indexing function [
Our model, as almost all models, depends on some hypotheses. Actually, it
depends on the following three hypotheses:
1. The more shared elements a document and a query have, the semantically
closer they are.
2. The descriptive power of an element (local weight): the more frequently an
element occurs in a document, the better it describes the document [
By taking these hypotheses into account, our model could be formulated. For any type of indexing elements the Relevance Status Value (RSV) between a document d and a query q is:
RSV (d; q) = kd \ qk (5) X N e2q Ne fd;e kdk ! Where d = feje 2 Index (d)g a document q = feje 2 Index (q)g a query kd \ qk = kfeje 2 Index (d) \ Index (q)gk the number of shared elements between a document d and a query q N the number of documents in the corpus Ne = kfdje 2 Index (d)gk the number of documents that contain the element e fd;e the number of occurances of an element e in a document d kdk the number of elements in a document d 6 Uni ed Medical Language System. It is a meta-thesaurus in medical domain.
http://www.ncbi.nlm.nih.gov/bookshelf/br.fcgi?book=nlmumls 2.3
Now after presenting our model in general, it should be instantiated according to each type of indexing elements.
RSVNG (d; q) = kd \ qkNG
X N ng2q Nng fd;ng kdkNG ! Where d = fngjng 2 IndexNG (d)g a document q = fngjng 2 IndexNG (q)g a query kd \ qkNG = kfngjng 2 IndexNG (d) \ IndexNG (q)gk the number of shared ngrams between a document d and a query q N the number of documents in the corpus Nng = kfdjng 2 IndexNG (d)gk the number of documents that contain the ngram ng fd;ng the number of occurances of a ngram ng in a document d kdkNG the number of ngrams in a document d
Keywords: We added to this instance a new component, which is the length
of keyword (the number of characters). Here we supposed that the longer a
keyword is, the more information it contains [
RSVK (d; q) = kd \ qkK Where kkk the number of characters in a keywords k
0 kdkK 1 kkkA RSVC (d; q) = kd \ qkC X N c2q Nc fd;c kdkC ! 2.4
As we said earlier, no single type of indexing elements could cover all aspects
of documents and queries. Therefore, merging all types (aspects) in one model
could enhance the performance of our model [
RSVall (d; q) = RSVNG (d; q) + RSVK (d; q) + RSVC (d; q) (9)
To increase the chance of retrieving more documents, another component could be added to the Formula (9). The component represents an expansion (6) (7) (8) of the query q. It is kd \ qkexpan: the number of shared keywords between a document d and a query q after replacing each query's concept c that does not occur in d by the set of keywords that represent c7.
RSVall (d; q) = RSVNG (d; q) + RSVK (d; q) + RSVC (d; q) + kd \ qkexpan (10) 3 3.1
In this task, image captions are used as documents and the English part of queries is just taken into account.
Text indexing: we extracted three types of indexing elements:
1. 5gram8: before extracting 5grams from documents and queries, we deleted
all non-ASCII characters. Then we used ve-characters-wide window for
extracting 5grams with shifting the window one character each time.
2. Keywords: before extracting keywords from documents and queries, we deleted
all non-ASCII characters. Then we eliminated the stop words and stem the
remaining keywords using Porter algorithm to get nally the list of keywords
that index documents and queries.
3. Concepts: before mapping the text of documents and queries to concepts,
we deleted all non-ASCII characters. Then we mapped the text to UMLS's
concepts using MetaMap [
Model variants: actually we experimented ve variants of our model in this task, which are:
RSVall (d; q) = RSV5G (d; q) + RSVK (d; q) + RSVC (d; q)
RSVall (d; q) = (kd \ qk5G;K;C ) (sum5G;K;C ) (11) (12)
RSVall (d; q) = RSV5G (d; q) + RSVK (d; q) + RSVC (d; q) + kd \ qkexpan (13) Where kd \ qk5G;K;C = kd \ qk5G + kd \ qkK + kd \ qkC sum5G;K;C = sum5G + sumK + sumC sum5G = P5g2q NN5g kfddk;55gG sumK = Pk2q NNk kfddk;kK sumC = Pc2q NNc kfddk;cC
RSVall (d; q) = (kd \ qk5G;K;C ) (sum5G;K;C + kd \ qkexpan) RSVall (d; q) = ((kd \ qk5G;K;C ) (sum5G;K;C )) + kd \ qkexpan (14) (15) 7 We supposed that each concept is represented by a set of keywords in its resource (we used UMLS as resource). 8 5gram is a ngram consists of ve characters. We picked out 5grams because they gave the best results using ImageCLEF2010 comparing to the other ngrams. 3.2 In this task, articles are used as documents. We indexed documents and queries in the same way as in the previous task. However, we extracted only two types of indexing elements (4grams, keywords) because of technical reasons.
Model variants: actually we experimented two variants of our model in this task, which are:
RSVall = RSV4G + RSVK RSVall (d; q) = (kd \ qk4G;K ) (sum4G;K ) (16) (17) Where kd \ qk4G;K = kd \ qk4G + kd \ qkK sum4G;K = sum4G + sumK sum4G = P4g2q NN4g kfddk;44gG sumK = Pk2q NNk kfddk;kK 4
Before we start representing and discussing the obtained results, we will present the names that we will use in our discussion for refering to each variant with its corresponding formula and run's name that used in the o cial campaign9 (see Table 1). 9 To see all results: http://www.imageclef.org/2011/medical (see Formula 5) to compute the matching value between a document and a query, we obtained good results. The run I CK5G 1 is ranked eighth out of 64 runs. That's because, we explicitly represented multi-facet (multi aspects) of documents and queries. In other words, three types of elements: 5Grams, Keywords, and Concepts are used and involved in the matching process.
From another side, using di erent fusion formulas to merge the results of using di erent types of indexing elements does not change a lot of things. Compare the results of the two runs: I CK5G 1 and I CK5G 2 (see Table 2).
In addition, the query expansion and the di erent formulas to integrate it into the model did not add anything. Compare the results of the three runs: I CK5G Q 1, I CK5G Q 2, and I CK5G Q 3 with the result of the run I CK5G 1 (see Table 2). That's because, the added value of query expansion is already compensated by using keywords and 5Grams.
Another noted result is that our model could retrieve the most relevant documents comparing to the other runs10. 4.2
The following table (see Table 3) contains the obtained results. The rst row (Best) is the result of the rst ranked run in the case-based retrieval task. 0.1500 0.1444 0.1278 # rel ret
Rank
Here also, we obtained good results. The run C K4G 1 is ranked fth out of 35 runs, knowing that, we used a very simple structure (set of elements), a 10 http://www.imageclef.org/2011/medical simple matching formula (see Formula 5), and also two simple types of indexing elements: Keywords and 4Grams. The two-facet (Keywords and 4Grams) representation of documents and queries was useful.
However, the formulas of merging the results of using di erent types of indexing elements were more sensitive comparing to the formulas in ad-hoc image-based retrieval task. Compare the results of the two runs: C K4G 1 and C K4G 2 (see Table 3). That's maybe because, we used less number of elements' types, two types (Keywords and 4Grams) in case-based retrieval comparing to three types (Concepts, Keywords, and 5Grams) in ad-hoc image-based retrieval. 5
We presented in this paper our approach to index and retrieve documents. We used three types of indexing elements (ngrams, keywords, concepts) for building a multi-facet document representation, and then we used a simple formula based on three hypotheses (the amount of overlap between a document and a query, the descriptive power of an indexing element, and the discriminative power of an indexing element) for retrieving documents, considering all facets (elements' types) of documents.
We obtained good results. The eighth out of 64 runs in the ad-hoc imagebased retrieval task, and the fth out of 35 runs in the case-based retrieval task, knowing that, we used a very simple structure for representing documents and queries and also a very simple ranking formula.