We reported some experiments conducted by our members in the SIG team at the IRIT laboratory in the CLEF medical retrieval task, namely ImageCLEFmed. In 2010, we are particularly interested in the case-based retrieval task. Our information retrieval approach integrates a hybrid method of concept extraction for enhancing the semantics of the document as well as of the query. More precisely, we applied a knowledge-based concept extraction method combined with statistical information obtained by scoring identified terms denoting concepts both in the document and query. The experiments carried out on the ImageCLEF 2010 show that our information retrieval approach based on the proposed method of concept extraction show an improvement of 7.07% in terms of MAP (for the best configuration trained on ImageCLEF 2010) over the baseline.
This paper describes the contribution of the SIG team1 (Generalized Information Systems) at the IRIT2 (Institute for Research in Informatics of Toulouse) laboratory in its first year participation at the medical retrieval track.
Started from 2004, the ImageCLEFmed (medical retrieval task) aims at
evaluating the performance of medical information systems, which retrieve medical
information from a mono or multilingual image collection, using visual features
and/or textual features. The ImageCLEFmed 2010 task consists of three
subtasks: modality classification, ad-hoc retrieval and case-based retrieval [
2 http://www.irit.fr patient cases with incomplete information consisting of patient demographics, symptoms, findings, test results and a set of images. The goal of this sub-task is to retrieve relevant cases that might best suit the provided case description. Motivated by the challenging characteristic of this sub-task, we are particularly interested in providing clinicians relevant information related to their requests.
The rest of this paper is organized as follows: Section 2 describes our conceptual indexing and retrieval framework, which integrates a hybrid approach of knowledge-based and statistical methods of concept extraction from medical documents as well as of the query. Identified terms denoting concepts extracted from the Medical Subject Headings3 thesaurus will be used to normalize the semantics of the documents (cases) or the queries (case topics). Submitted results will be presented and discussed in section 3. We conclude the paper in section 4 by outlining some perspectives for future work. 2
The conceptual indexing and retrieval framework consists of three main components: (1) concept extractor, (2) conceptual indexer and (3) conceptual retriever. 2.1
Our concept extractor relies on a knowledge-based and statistical concept extraction method. Given a patient case, which is typically a textual document including title and image captions, medical terms denoting MeSH concepts are firstly recognized using MeSH lexicon4. The concept extraction is processed through three steps: (1) pre-processing, (2) term recognizer and (3) term weighting.
In the pre-processing step, original documents are aggregated by two parts,
namely title and image captions, from each unique article. Documents are then
converted into the TREC-like format. During the main processing step, each
document is splitted into sentences using TreeTagger [
4 MeSH lexicon contains all meaningful terms (preferred or non-preferred terms) in the thesaurus
Many researchers think that IR techniques could be used to extract
technical terms denoting concepts for conceptual indexing purposes. However, most
of works dealing with IR techniques for concept extraction are based on
wordbased representations. For instance, recent works such as [
We hypothesize that a MeSH concept can be thought of as a document containing biomedical terms describing itself. Each concept in the MeSH thesaurus, which can be distinguished from others by its concept unique identifier (CUI), contains many textual fields such as: MAIN HEADING (concept name or preferred term), ENTRY (synonyms or lexical variants or non-preferred terms), QUALIFIERS, SCOPE NOTE etc. Different synonyms and lexical variants of this concept could be found in the ENTRY field.
Here, we are mainly interested by concept entries (MAIN HEADING,
ENTRY) since they constitute the most common indexing and retrieval features
used in the domain. Let’s denote Entries(C) the set of preferred and not
preferred terms of concept C. According to our approach, MeSH thesaurus is viewed
as a collection of textual concepts. Formally, each concept Ci of the MeSH
thesaurus is represented as a vector of linearly basis vectors namely basic terms
in the MeSH lexicon: C = (c1, c2, . . . cNc) where N c is MeSH lexicon size, cj
is a weight measuring the aboutness of term cj in a document D, computed
according to the BM25 weighting schema [
tfcCj ∗ (k3 + 1) ∗ tfcDj (k3 + tfcDj ) ∗ k1∗(1−b)+b∗cl avgcl+tfcCj ∗ log
Nc − nj + 0.5 nj + 0.5 (1) where tfcCj is the number of occurrences of term cj in concept C, Nc is the total number of concepts in MeSH, nj is the number of concepts containing term cj, cl is the concept length of C (i.e. total number of distinct terms occurring in its textual features), and avcl is the average concept length in MeSH, k1 = 1.2, k3 = 8, b = 0.75 are the constants used in the experiments reported here.
We applied the BM25 weighting model to measure the degree of
expressiveness of each recognized terms (both multi-word and single-word terms) denoting
concepts. In such as a way, our concept extraction approach is typically based on
the combination of both the knowledge-based and statistical based methods,
allowing to recognize a list of candidate terms denoting concepts in the document
that are ranked in an decreasing order of their ability to describe the document.
Given a list of recognized terms denoting concepts in the document, each of
them is assigned by a score based on the the state-of-the-art term scoring
function BM25 [
Inspired by recent works dealing with medical concept extraction for
document and query expansion [
The retriever component aims at finding the most relevant documents (search results) in response to a user query. At this stage, documents are retrieved and ranked on the basis of a relevance estimation, which is usually incorporated into a term weighting model (e.g., TF-IDF, PL2, BM25 ...). We used Terrier with appropriate settings (described later in section 3) to perform the retrieval. In such settings, documents and eventually queries are expanded with concept names (or preferred terms) identified by our hybrid concept extraction method. The relevance score of the document Di with respect to the query Q is given by:
RSV (Q, Di) = RSV (Qw, Diw) + RSV (Qc, Dic) where RSV (Qw, Diw) is the TF-IDF word-based relevance score and RSV (Q, Ds) i is the concept-based relevance score of the document w.r.t the query, computed as follows:
RSV (Qw, Diw) = Pqkw∈Qw (1 + αw) ∗ T Fi(qkw) ∗ IDF (qkw)
RSV (Qc, Dic) = Pqkc∈Qc (1 + αc) ∗ T Fi(qkc ) ∗ IDF (qkc ) where T Fi: the normalized term frequency of the word qkw or preferred term qkc in document Di, IDF : the normalized inverse document frequency of qkw or qkc in the collection, αw: the word score modifier, αc: the preferred term score modifier. The values of the parameter α are obtained by training the retrieval on an IR benchmark. (2) (3) 3
The goal of our experiments is to evaluate the retrieval effectiveness based on our concept extraction method as well as the impact of the document expansion (DE) and query expansion (QE) using an appropriate number of preferred terms. Terms appearing in a specific field may have a different relevance score to others. Title6, image captions7 and kernel8 are the three main fields of the document. 6 article title of the patient case 7 aggregated text obtained by combining all image captions in a patient case 8 the expanded preferred terms to the document
We carried out two sets of experiments: the first one is based on the classical
index of titles and image captions of patient cases using Terrier standard
configuration based on the state of the art weighting scheme OKAPI BM25 [
We use both terms representing MeSH concepts (main headings or preferred terms) and single words that do not match any entry in the thesaurus. In the classical approach, documents, i.e. patient cases, were first indexed using the Terrier IR platform (http://ir.dcs.gla.ac.uk/terrier/). It consists in processing single words occurring in the documents through a pipeline: removing stop words, and stemming9 of English words.
In our conceptual IR approach for case-based retrieval, documents and/or
queries are firstly analyzed to extract an appropriate number of concepts, namely
N and indexed with an appropriate term weighting schema. The parameter
N is an experimental variable that must be learned from an IR benchmark
by regarding the MAP value or a MEDLINE sub-collection by regarding the
F-measure. It very depends on the IR/concept extraction performance of the
underlying system. Through some experiments on the ImageCLEFmed 2009 [
DE+QE+field 3 +7.07%
Table 2: Improvement rates over the baseline This article describes the conceptual retrieval approach of the SIG team for the ImageCLEF 2010 medical retrieval track, especially the case-based retrieval task. The results obtained by our submitted runs prove that our method of concept extraction is useful to enhance the semantics of the document, which could be an interesting evidence to improve the retrieval effectiveness of medical retrieval systems. However, the retrieval performance can be better improved by state-of-the-art query expansion techniques.