=Paper=
{{Paper
|id=Vol-1178/CLEF2012wn-ImageCLEF-AbdulahhadEt2012
|storemode=property
|title=MRIM at ImageCLEF2012. From Words to Concepts: A New Counting Approach
|pdfUrl=https://ceur-ws.org/Vol-1178/CLEF2012wn-ImageCLEF-AbdulahhadEt2012.pdf
|volume=Vol-1178
}}
==MRIM at ImageCLEF2012. From Words to Concepts: A New Counting Approach ==
https://ceur-ws.org/Vol-1178/CLEF2012wn-ImageCLEF-AbdulahhadEt2012.pdf
MRIM at ImageCLEF2012. From Words to
Concepts: A New Counting Approach
Karam Abdulahhad*, Jean-Pierre Chevallet**, and Catherine Berrut*
* UJF-Grenoble 1, ** UPMF-Grenoble 2, LIG laboratory, MRIM group
karam.abdulahhad,jean-pierre.chevallet,catherine.berrut@imag.fr
Abstract. MRIM research group has participated in two tasks (ad-hoc
image-based retrieval and case-based retrieval) of the ImageCLEF2012
Medical Retrieval track. In our contribution, we study the frequency shift
problem that happens when using concepts instead of words as indexing
terms. The main goal of our experiments is to check the validity of our
new counting strategy of concepts (Relative Count), which is proposed
as a solution to the frequency shift problem. In order to validate our new
counting strategy, we compare the retrieval performance (represented by
MAP) of some classical IR models using the classical counting strategy
(count each concept as 1) with their performance using the new strategy.
The results are promising, and using the new counting strategy shows a
considerable gain in performance. We use in our experiments two sup-
plementary resources: MetaMap as a text-to-concepts mapping tool, and
UMLS as an external resource containing concepts.
1 Introduction
We present in this paper the contribution of the MRIM1 research group in the
ImageCLEF20122 Medical Image Retrieval task3 .
The main goal of our experiments and contribution is to study the side-effects
of moving from the word-space to the concept-space on the classical Information
Retrieval (IR) models. In other words, to study the side-effects of using concepts
instead of words as indexing terms on the classical IR models. Concepts are
entry IDs in a specific external resource, and each concept is associated to a set
of strings that describe it.
However, in order to build a concept-based IR system, another component,
for mapping documents and queries text into concepts, is needed. These mapping
tools, e.g. MetaMap4 [2], are imperfect, and they could map one piece of text
1
Multimedia Information Modeling and Retrieval is a research group in LIG (Labo-
ratoire d’Informatique de Grenoble) laboratory.
http://mrim.imag.fr/
http://www.liglab.fr/
2
http://www.imageclef.org/2012
3
http://www.imageclef.org/2012/medical
4
http://metamap.nlm.nih.gov/
into no or several candidate concepts. For example, MetaMap maps a text like
”x-ray” into six different UMLS5 concepts.
Therefore and as classical IR models directly or indirectly depend on the
shared terms d ∩ q between a document d and a query q in order to compute the
Relevance Status Value RSV (d, q) [4], we have what we called a frequency shift
problem, because the number of shared terms |d ∩ q| between d and q changes
in a non-homogeneous way when moving from the word-space to the concept-
space. For example, using a mapping tool like MetaMap and an external resource
like UMLS, if d and q share one word ”x-ray” in the word-space, then they will
share six different concepts in the concept-space. Whereas, if they share a noun-
phrase of two words ”lung x-ray” in the word-space, then they will share only
one concept ”C0581647 ” in the concept-space.
One solution to this problem is the supplementary disambiguation step that
is actually achieved alongside the mapping process [3]. However, in this study we
follow another strategy that proposes another concept counting mechanism. We
do not count a concept as 1, instead of that, we give to each concept a relative
count (in R∗+ ) respecting the following two hypothesis:
– concepts that correspond to a longer text should receive larger count.
– the count of a concept should be inversely proportional to the ambiguity level
of its corresponding text. The ambiguity level of a piece of text is determined
by the number of concepts that is mapped into. The text that is mapped
into a larger number of concepts is more ambiguous.
The goal is to finally satisfy the following condition for a piece of text TXT :
X
|TXT | = count c (1)
c∈map(TXT )
where, |TXT | is the number of words in TXT , map(TXT ) is the set of all
candidate concepts of TXT , and count c ∈ R∗+ is the new relative count of the
concept c and we will explain in the following sections the algorithm of computing
it.
This year, ImageCLEF2012 contains four main tracks: 1) Medical Image
Classification and Retrieval, 2) Photo Annotation and Retrieval, 3) Plant Iden-
tification, and 4) Robot Vision. Medical Image Classification and Retrieval track
contains three tasks: 1) modality classification, 2) ad-hoc image-based retrieval
which is an image retrieval task using textual, image or mixed queries, and 3)
case-based retrieval: in this task the documents are journal articles extracted
from PubMed6 and the queries are case descriptions. We participated in the last
two tasks: ad-hoc image-based retrieval and case-based retrieval. Table 1 shows
some statistics on the data collections of the two tasks that we participated in.
We only use the textual data.
This paper is structured as follows: Section 2 describes in details the problem
that occurs when moving from the word-space to the concept-space, and it also
5
Unified Medical Language System (http://www.nlm.nih.gov/research/umls/)
6
http://www.ncbi.nlm.nih.gov/pubmed/
Table 1. Corpora statistics. avdl and avql are the average length of documents and
queries. ’C’ for concepts and ’W ’ for words. ’image2012’ for ad-hoc image-based re-
trieval data collection and ’case2012’ for case-based retrieval data collection.
Corpus #d #q Used fields Type avdl avql
W 47.16 3.55
image2012 306530 22 -
C 104.26 9.41
W 160.51 24.35
title+abstract
C 376.14 63.73
case2012 74654 26
W 2731.24.35 24.35
title+abstract+fulltext
C - -
presents our proposed solution. Section 3 presents all technical details of applying
our proposed solution to ImageCLEF2012 test collections. It also shows our
formal runs and the obtained results. We conclude in section 4.
2 Computing Relative Concept Count
Our algorithm depends on the output of MetaMap as a mapping tool and on the
UMLS as an external resource containing the concepts. However, the algorithm
could be easily generalized because most mapping tools [2][5] have the same
general text-to-concept mapping mechanism [3].
For a textual document d or a query q, mapping tools (e.g. MetaMap) extract
noun-phrases from the text and try to map them into one or more candidate
concepts of a specific external resource (e.g. UMLS).
However, for a noun-phrase np, it is sometimes difficult to find concepts
corresponding to the whole noun-phrase. Moreover, even if there are concepts
corresponding to the whole noun-phrase np, it is useful to return some concepts
corresponding to parts of np, because restricting our attention to the concepts
that only correspond to the whole phrase could lead to miss some related con-
cepts, or in other words, it could lead to lose in recall. Therefore, most mapping
tools do not only depend on the exact match to find candidate concepts, but
they also generate some variants7 of the original noun-phrase [2], and then find-
ing candidate concepts of all variants instead of only the original noun-phrase.
For example, Table 2 shows the variants of the noun-phrase ”lobar pneumonia
x-ray” that are generated by MetaMap, their related candidate UMLS concepts,
and the corresponding part of the original noun-phrase.
In this study, we regroup all variants that correspond to the same part of the
original noun-phrase into only one variant. Therefore, Table 2 become Table 3.
7
spelling variants, abbreviations, acronyms, synonyms, inflectional and derivational
variants, or meaningful combinations of these.
Table 2. Variants of ”lobar pneumonia x-ray” generated by MetaMap, their related
candidate UMLS concepts, and the corresponding part of the original noun-phrase
Variants Candidate concepts Corresponding part
”lobar pneumonia x-ray” - ”lobar pneumonia x-ray”
”lobar pneumonia” C0032300, C0155862 ”lobar pneumonia”
”lung x-ray” C0581647 ”pneumonia x-ray”
”lung” C0024109, C 1278908 ”pneumonia”
”pneumonia” C0032285 ”pneumonia”
”pulmonary” C2707265, C2709248 ”pneumonia”
”lobar ” C1522010 ”lobar ”
”lobe” C1428707 ”lobar ”
”lobus” C0796494 ”lobar ”
C0034571, C0043299, C0043309
”x-ray” ”x-ray”
C1306645, C1714805, C1962945
Table 3. Variants of ”lobar pneumonia x-ray” after regrouping according to the cor-
responding parts of the original noun-phrase
Variants Candidate concepts
”lobar pneumonia x-ray” -
”lobar pneumonia” C0032300, C0155862
”pneumonia x-ray” C0581647
C0024109, C1278908, C0032285
”pneumonia”
C2707265, C2709248
”lobar ” C1522010, C1428707, C0796494
C0034571, C0043299, C0043309
”x-ray”
C1306645, C1714805, C1962945
2.1 Definitions
Our algorithm locally works at the level of noun-phrases not at the level of
documents.
Each noun-phrase np is a sequence of words or a set of 2-tuples, where each
tuple (w, i) contains a word w ∈ W and the position i ∈ N∗ of w in np. Any
variant v of np is also supposed to be a noun-phrase. By this way, it is possible
to attach to each noun-phrase np, a set Vnp :
Vnp = {np} ∪ {v1 , . . . , vj }
where, {v1 , . . . , vj } are the variants of np that are generated by the mapping
tool.
We define the function wrd that returns the set of word-position tuples of a
specific variant.
∗
wrd : Vnp → 2W ×N
For example, suppose the variant v =”pneumonia x-ray” of the phrase ”lobar
pneumonia x-ray”, then:
wrd (v) = {(pneumonia, 2), (x − ray, 3)}
We also define for any variant v ∈ Vnp , |v| = |wrd (v)| the number of words in v.
We define the function map that returns the candidate concepts of a variant
v ∈ Vnp .
map : Vnp → 2C
where C is a set of concepts. For example, suppose the variant v =”pneumonia
x-ray” of the phrase ”lobar pneumonia x-ray”, then:
map(v) = {C0581647}
We will remove from Vnp all members v that do not have any candidate
concepts map(v) = φ. For example, in the case of ”lobar pneumonia x-ray”
noun-phrase, we will remove the noun-phrase itself because it is not mapped
into any concept. Finally, Vnp becomes:
Vnp ={”lobar pneumonia”, ”pneumonia x-ray”, ”pneumonia”, ”lobar ”, ”x-ray”}
It is possible to define a partial order < relation on the set Vnp as follow:
∀v1 , v2 ∈ Vnp , v1 < v2 iff wrd (v1 ) ⊂ wrd (v2 )
Therefore, it is possible to define a hierarchy HR np on Vnp ∪ {R}, where:
– v ∈ Vnp ∪ {R} are the nodes.
– R is an abstract root satisfying: ∀v ∈ Vnp , v < R. Moreover, |R| = 0.
– The direct children ch(v) of any node v ∈ Vnp ∪ {R} is defined as follow:
∀v1 , v2 ∈ Vnp ∪ {R},
v1 ∈ ch(v2 ) iff v1 < v2 and 6 ∃v3 ∈ Vnp , v1 < v3 < v2
– The direct parents pr (v) of any node v ∈ Vnp ∪ {R} is defined as follow:
∀v1 , v2 ∈ Vnp ∪ {R}, v1 ∈ pr (v2 ) iff v2 ∈ ch(v1 )
For example, Fig. 1 shows the hierarchy of the noun-phrase ”lobar pneumonia
x-ray”.
2.2 The Algorithm
The main goal of the algorithm is to compute the relative count of candidate
concepts, as follow: for a noun-phrase np, distributing the number of words
|np| of this noun-phrase on the candidate concepts of np and its variants. The
algorithm respects two hypothesis:
– the relative count of a concept is directly proportional to the number of
words in the corresponding variant.
Fig. 1. The hierarchy of the noun-phrase ”lobar pneumonia x-ray”.
– the relative count of a concept is inversely proportional to the ambiguity
level of the corresponding variant. The variant that is mapped into a larger
number of concepts is more ambiguous.
The input is the set Vnp that contains all variants of np, and the number
of words |np| in np. The output will be a set CC np of 2-tuples, and each tuple
contains one of the candidate concepts and its associated relative count. The
detailed algorithm is shown in Algorithm 1.
2.3 Example
Assume np is the noun-phrase ”lobar pneumonia x-ray”. The list of variant Vnp
is: Vnp = {v1 , v2 , v3 , v4 , v5 } (see Fig. 1). Figure 1 also shows the hierarchy HR np
of the set Vnp ∪ {R}. Initially, we have αR = |np| = 3 and ∀v ∈ Vnp , αv = 0. We
scan the nodes in breadth-first order, that means, nodes will be scanned in the
following order < R, v1 , v2 , v3 , v4 , v5 >.
By following the Algorithm P 1, Table 4 shows the output set CC np . By con-
sulting Table 4, we can see: (c,count c )∈CC np count c = |np| = 3.
3 Experiments
The main goal of our experiments is to show the validity of our new way of
counting. We check this validity through comparing the retrieval performance
of some classical IR models using classical concept counting method, to their
performance using our new way of counting.
We use the Mean Average Precision (MAP) metric as an indicator to the
retrieval performance of IR models.
3.1 Retrieval Models
In order to check the validity of our new way of counting, we use several classical
IR models:
Algorithm 1: RelativeCount
input : Vnp , |np|
output: CC np ⊆ C × R∗+
1 CC np = {};
2 construct the hierarchy HR np of the set Vnp ∪ {R};
0
3 attach to each node n in HR np a value αn0 , where αn0 is the total amount that
0
is received from pr (n ) and should be distributed on the candidate concepts of
n0 and its children ch(n0 ), starting by αn0 = 0;
4 n is the current node, starting by n = R;
5 set αR = |np|;
6 scan HR np in a breadth-first way, starting from the current node n = R;
7 begin
8 for each child n0i ∈ ch(n) do
9 compute the amount αn0i that should be transferred from n to n0i :
α ×|n0i |
αn0i = αn0i + |n|+P n0 0 ;
n ∈ch(n) |nj |
j
10 end
11 if n 6= R then
12 compute the amount an that should be distributed on the candidate
concepts map(n) of the current node n: an = |n|+Pαn0 ×|n| |n0 | ;
n ∈ch(n) j
j
13 for each candidate concept ci ∈ map(n) do
14 compute the relative count: count np an
ci = |map(n)| ;
np
15 CC np = CC np ∪ {(ci , count ci )};
16 end
17 end
18 change n to the next node according to the breadth-first scan order;
19 go to line 7;
20 end
– from probabilistic framework: we choose BM25 (2) [7].
– from language models framework: we choose Dirichlet model DIR (3) [9],
and Jelinek-Mercer model JM (4) [9].
– from vector space framework: we choose Pivoted Normalization Method PIV
(5) [8], and a version of TFIDF model (6) [1].
where, tf t,d is the term frequency of the indexing term t in the document d, tf t,q
is the term frequency of the indexing term t in the query q, |d| is the document
length, |q| is the query length, avdl is the average document length, N is the
total number of documents in the corpus D, nt is the number of documents that
contain the indexing term t, d ∩ q are the shared indexing terms between d and
q, and p(t, D) is the probability of t given the corpus language model D.
s, k1 , b, k3 , λ, and µ are all parameters. They usually have the following
values: s = 0.2 [8]. k1 = 1.2, b = 0.75, and k3 = 1000 [6]. λ = 0.1 for short
queries or λ = 0.7 for long queries and µ = 2000 [9].
Table 4. The output set CC np of the noun-phrase ”lobar pneumonia x-ray”
Vnp CC np
v1 =”lobar pneumonia” (C0032300,0.375), (C0155862,0.375)
v2 =”pneumonia x-ray” (C0581647,0.75)
v3 =”lobar ” (C1522010,0.125), (C1428707,0.125), (C0796494,0.125)
(C0024109,0.15), (C1278908,0.15), (C0032285,0.15)
v4 =”pneumonia”
(C2707265,0.15), (C2709248,0.15)
(C0034571,0.0625), (C0043299,0.0625), (C0043309,0.0625)
v5 =”x-ray”
(C1306645,0.0625), (C1714805,0.0625), (C1962945,0.0625)
P N −nt +0.5 (k1 +1)×tf t,d
RSV (d, q) = t∈d∩q ln nt +0.5 × k1 ×((1−b)+b× |d| )+tf
avdl t,d
(2)
(k +1)×tf
× 3k3 ×tf t,q
t,q
tf t,d
µ X
RSV (d, q) = |q| × ln + tf t,q × ln 1 + (3)
|d| + µ µ × p (t, D)
t∈d∩q
tf t,d
X 1−λ
RSV (d, q) = |q| × ln(λ) + tf t,q × ln 1 + × (4)
λ |d| × p (t, D)
t∈d∩q
X 1 + ln 1 + ln tf t,d N +1
RSV (d, q) = |d|
× tf t,q × ln (5)
(1 − s) + s avdl nt
t∈d∩q
X tf t,d N
RSV (d, q) = |d ∩ q| × tf t,q × |d|
× (6)
tf t,d + avdl nt
t∈d∩q
There are two different views for documents and queries. The first one is the
classical view, where both documents and queries are bags of indexing terms
(e.g. words or concepts). The classical way of counting (count each concept
or word as 1) is compatible with this view, and the previous definitions of IR
model components correspond to this classical view. We will present the second
view, which is convenient to the relative count of concepts. In this view, some
components of IR models should be redefined.
Most mapping tools extract noun-phrases from the text of documents and
queries. Therefore, any document d or query q is a sequence of noun-phrases:
d =< np1 , . . . , npnd > q =< np1 , . . . , npnq >
We redefine the set CC np on the level of documents and queries instead of the
noun-phrase level:
[ [
CC d = CC npi CC q = CC npi
npi ∈d npi ∈q
CC d∩q = {(c, r)|(c, r) ∈ CC q , ∃(ci , ri ) ∈ CC d , c = ci }
According to our way of counting (Relative Count), the components of the IR
models become:
X X X
tf c,d = ri tf c,q = ri tf c,D = tf c,di
(c,ri )∈CC d (c,ri )∈CC q di ∈D
X X X
|d| = ri |q| = ri |D| = |di |
(ci ,ri )∈CC d (ci ,ri )∈CC q di ∈D
X
|d ∩ q| = ri
(ci ,ri )∈CC d∩q
3.2 Data
We only use the textual part of the data collections of ad-hoc image-based re-
trieval and case-based retrieval tasks of the ImageCLEF 2012 Medical Image
Classification and Retrieval track. Table 1 shows some statistics about these two
data collections.
We use two types of indexing terms:
– words (W): we eliminate the stop words and stem the remaining words using
Porter algorithm to finally get the list of words that indexes documents and
queries.
– concepts (C): we use MetaMap for mapping the documents and queries text
content into UMLS concepts.
3.3 Formal Runs
The initial of our runs in the formal campaign is ”LIG MRIM xxx”. However,
Tables 5 and 6 show the name and the description of our runs. The best MAP in
the ad-hoc image-based retrieval task (only text) is 0.2182, and the best MAP
in the case-based retrieval task (text only) is 0.1690.
3.4 Results and Discussion
In this section we show the validity of our new method of concept counting
(RelativeCount), through comparing the retrieval performance of IR models with
or without using the relative count. The only type of indexing terms that is used
in this section is concepts. We count concepts in two ways: the classical count
(count each concept as 1) and the relative count (Algorithm 1).
Table 7 shows the results of applying IR models to the image2012 collection
(see Teble 1).
Table 8 shows the results of applying IR models to the case2012 collection
(see Teble 1). We only map the title and the abstract parts of documents into
concepts
Table 5. Our formal runs in the ad-hoc image-based retrieval task. (TFIDF*) means
the TFIDF model after removing |d ∩ q|. (TFIDF**) means the TFIDF model after
removing |d ∩ q| and avdl
run name IR model term type count type MAP
IB TFIDF W avdl DintQ TFIDF W - 0.1586
IB FUSION TFIDF W TB C avdl DintQ TFIDF W+C relative 0.1432
IB FUSION JM01 W TB C JM(λ = 0.1) W + C relative 0.1425
IB TB PIVv2 C PIV C relative 0.1383
IB TFIDF C avdl DintQ TFIDF C classic 0.1345
IB TB JM01 C JM(λ = 0.1) C relative 0.1342
IB TB BM25 C BM25 C relative 0.1165
IB TB TFIDF C avdl TFIDF* C relative 0.1081
IB TB DIR C DIR C relative 0.0993
IB TB TFIDF C TFIDF** C relative 0.0900
Table 6. Our formal runs in the case-based retrieval task. (TFIDF*) means the TFIDF
model after removing avdl. (TFIDF**) means the TFIDF model after removing |d ∩ q|.
(TFIDF***) means the TFIDF model after removing |d ∩ q| and avdl. The concepts
are extracted from the title and abstract of each document
run name IR model term type count type MAP
CB FUSION DIR W TA TB C DIR W+C relative 0.1508
CB FUSION JM07 W TA TB C JM(λ = 0.7) W + C relative 0.1384
CB TFIDF W DintQ TFIDF* W - 0.1036
CB TA TB JM07 C JM(λ = 0.7) C relative 0.0908
CB TA TB BM25 C BM25 C relative 0.0895
CB TA TB DIR C DIR C relative 0.0893
CB TA TB PIVv2 C PIV C relative 0.0865
CB TA TFIDF C DintQ TFIDF* C classic 0.0789
CB TA TB TFIDF C avdl TFIDF** C relative 0.0692
CB TA TB TFIDF C TFIDF*** C relative 0.0646
Tables 7 and 8 show that we have a considerable gain in retrieval performance
when using the relative count instead of the classical one. The gain is clearer
in the case of short documents and queries (image2012) than the case of long
documents and queries (case2012).
Tables 5 and 6 show that using words as indexing terms is still more effective
(from the retrieval performance point of view) than using concepts.
Concerning our formal contribution in the ad-hoc image-based retrieval task,
we got a middle-rank. We still far from the best formal run. However, our con-
tribution in the case-based retrieval task was more encouraging. We are ranked
the second in the final list. We made a late fusion between two result set: 1- one
resulting from applying DIR model to the case2012 corpus using words as index-
ing terms, and 2- another one resulting from applying DIR model to case2012
corpus using concepts as indexing terms.
Table 7. Comparing the retrieval performance of IR models using the classical count
with their performance using the relative count (Algorithm 1). The corpus is image2012.
The last column shows the gain in performance when using the relative count comparing
to the performance of using the classical count
IR model count type MAP Gain
classic 0.1071
PIV +27%
relative 0.1360
classic 0.1034
BM25 +10%
relative 0.1142
classic 0.0861
DIR +13%
relative 0.0969
classic 0.1022
JM(λ = 0.1) +29%
relative 0.1318
classic 0.1322
TFIDF +7%
relative 0.1410
4 Conclusion
We present in this paper the contribution of the MRIM research group in the
ImageCLEF2012 Medical Image Retrieval task. We describe the frequency shift
problem that happens when moving from the word-space to the concept-space.
The source of this problem is the heterogeneous change in the frequency of
indexing terms when moving from the word-space to the concept-space.
We propose a solution to the frequency shift through a new counting strategy.
Our counting strategy (Algorithm 1) depends on the hierarchy that could be
built from the output of mapping tools. It also depends on the following two
hypotheses:
– the relative count of a concept is directly proportional to the number of
words in the corresponding text.
– the relative count of a concept is inversely proportional to the ambiguity
level of the corresponding text.
For validating the effectiveness (from the retrieval performance point of view)
of our new counting strategy, we participated in the ImageCLEF2012 campaign,
more precisely, in the ad-hoc image-based retrieval and case-based retrieval tasks.
Our experiments only depend on the textual data. For mapping text into con-
cepts, we use MetaMap as a mapping tool and UMLS as an external resource
containing concepts. In the case-based retrieval task, we only map the title and
abstract of each document (we do not map the fulltext).
In the ad-hoc image-based retrieval task, we got a middle-rank. We still far
from the best formal run. However, our contribution in the case-based retrieval
task was more encouraging. We are ranked the second in the final list.
Moreover, the supplementary results that we present in this paper show a
considerable gain in retrieval performance when applying our counting strategy
Table 8. Comparing the retrieval performance of IR models using the classical count
with their performance using the relative count (Algorithm 1). The corpus is case2012.
The last column shows the gain in performance when using the relative count comparing
to the performance of using the classical count
IR model count type MAP Gain
classic 0.0789
PIV +10%
relative 0.0865
classic 0.0847
BM25 +7%
relative 0.0895
classic 0.0825
DIR +8%
relative 0.0893
classic 0.0863
JM(λ = 0.7) +5%
relative 0.0908
classic 0.0830
TFIDF +2%
relative 0.0847
(Algorithm 1) comparing to the classical counting strategy (count each concept
as 1).
References
1. Karam Abdulahhad, Jean-Pierre Chevallet, and Catherine Berrut. The Effective
Relevance Link between a Document and a Query. In 23rd International Conference
on Database and Expert Systems Applications (DEXA 2012), Vienna, Austria, pages
206–218, sep 2012.
2. Alan R. Aronson. Metamap: Mapping text to the umls metathesaurus, 2006.
3. Jean-Pierre Chevallet, Joo Hwee Lim, and Thi Hoang Diem Le. Domain knowledge
conceptual inter-media indexing, application to multilingual multimedia medical
reports. In ACM Sixteenth Conference on Information and Knowledge Management
(CIKM 2007), Lisboa, Portugal, November 6–9 2007.
4. Stéphane Clinchant and Eric Gaussier. Information-based models for ad hoc ir.
In Proceedings of the 33rd international ACM SIGIR conference on Research and
development in information retrieval, SIGIR ’10, pages 234–241, New York, NY,
USA, 2010. ACM.
5. Christopher Dozier, Ravi Kondadadi, Khalid Al-Kofahi, Mark Chaudhary, and
Xi Guo. Fast tagging of medical terms in legal text. In Proceedings of the 11th
international conference on Artificial intelligence and law, ICAIL ’07, pages 253–
260, New York, NY, USA, 2007. ACM.
6. Hui Fang, Tao Tao, and ChengXiang Zhai. A formal study of information retrieval
heuristics. In Proceedings of the 27th annual international ACM SIGIR conference
on Research and development in information retrieval, SIGIR ’04, pages 49–56, New
York, NY, USA, 2004. ACM.
7. S. E. Robertson and S. Walker. Some simple effective approximations to the 2-
poisson model for probabilistic weighted retrieval. In Proceedings of the 17th annual
international ACM SIGIR conference on Research and development in information
retrieval, SIGIR ’94, pages 232–241, New York, NY, USA, 1994. Springer-Verlag
New York, Inc.
8. Amit Singhal, Chris Buckley, and Mandar Mitra. Pivoted document length normal-
ization. In Proceedings of the 19th annual international ACM SIGIR conference on
Research and development in information retrieval, SIGIR ’96, pages 21–29, New
York, NY, USA, 1996. ACM.
9. Chengxiang Zhai and John Lafferty. A study of smoothing methods for language
models applied to ad hoc information retrieval. In Proceedings of the 24th annual
international ACM SIGIR conference on Research and development in information
retrieval, SIGIR ’01, pages 334–342, New York, NY, USA, 2001. ACM.