This paper presents our rst participation in user-centred health information retrieval task at the CLEFeHealth 2014. This task has as objective the information retrieval to answer patients' questions when reading clinical reports. We have submitted only the mandatory run (baseline system). The obtained results are motivating with map=0:1677 and p@10=0:5460 but can be improved.
With the increasing number of documents on the web, searching relevant documents to users queries becomes a difficult task especially if the queries are short and do not represent well the user need. The process of searching documents in a corpus in order to meet a need is called Information Retrieval (IR). This process (IR) focuses on the representation, storage, organization of information that should allow the user quick and easy access to information. To automate the task of RI, Information Retrieval Systems (IRS) are developed to provide all necessary functions for information retrieval illustrates in the figure 1.
The system builds an index of the documents which is an essential data structure because it allows fast searching over large volumes of data.
Given that the document database is indexed, the searching process can be started. The user specifies a query which is parsed and transformed by the same operations applied to the document database. After that the system retrieves documents that are relevant to the query from the index and displays them to the user with a ranked way.
The user can examine the ranked documents. He can identify a subset of the
documents seen as relevant and initiate a relevance feedback step. The system
uses the documents selected by the user in order to refine the query.
The evaluation of such systems appears to be a necessity. This evaluation is
based on the notion of relevance [
{ Boolean Model:
It is a model which is based on the set theory. Thus, the user query is represented as a logical equation composed of keywords and Boolean (logic) operators (OR, AND, NOT) and documents are represented by a list of keywords. The research process with this model performs operations of union, intersection and difference, defined by the existence or absence of index terms, to achieve an exact match between the documents and the equation of the query. { Probabilistic model:
The probabilistic model addresses the problem of information retrieval in a probabilistic framework. It allows modeling of the selection process documents in an IRS based on probability theory. The basic principle of the probabilistic model is to present the search results in an IRS order (O) based on the probability of relevance of a document with respect to a query. The basic idea is first to calculate two conditional probabilities P (R=D) and P (N R=D) with a given query R is the relevance (all relevant documents) and N R irrelevance (all irrelevant documents). The terms are not weighted, but only take the value 0 (term absent) or 1 (this term). The principle of the model is to calculate an ordering function (O(D)) that would classify documents that meet a query with { Vector Space Model:
The vector space model (VSM) was developed by Salton for the SMART
information retrieval system [
In our participation, we used Vector Space Model because it use the weighted vector not the binary one and it allows computing a degree of similarity between documents and queries; also it returns ranking documents according to their relevance. 2
2.1
Document Collection: The data set for the ehealth information retrieval task
consists of a set of medical-related documents, provided by the Khresmoi project
[
Topics: The test set consists of 50 medical professional queries containing the
following fields [
Our work was to index and search the top-1000 relevant medical documents for each topic using a plain-text search engine.
We chose to use the terrier platform in our case [
Stemming: is a process of linguistic normalization for reducing variant forms of tokens to their stem or root form.
Storing: information (terms) were stored in file with special structure called inverted file by specifying the number of occurrences (term frequency (TF) and their location. This file allows rapid access during query time.
The third step is searching relevant documents for a particular query; it was carried online. Firstly, we processed query in the same way documents were processed during indexing. Then we represented documents and query with weighted vectors of terms which represents the degree of relationship between a term and a document in the whole collection. This degree is obtained by multiplying the two measures (TF) and (IDF) with
IDFtermi = log
j D j j dj : termi 2 dj j (2) Where j D j : total number of documents in the collection. j dj : termi 2 dj j: number of documents where the termi appears Then we calculated similarity (scores) between vectors using cosine similarity measure and ranked the documents according to their scores in descending order. The top 1000 documents with the highest scores were the best match to the query and returned as relevant documents.
In table 1 we show results obtained by taking account of pooling all search results.
Our system can retrive 38% (1189 among 3209) of relevant documents and has a measure of map equal to 0:1677 and of p@10 equal to 0:5460.
These results are motivating but can be improved by including for example the
notion of semantics in the search process [
The plots below in figure 2 compares our run against the median and best performance (p@10) across all systems submitted to Ehealth task for each query. In particular, for each query, the height of a bar represents the gain/loss of your system and the best system (for that query) over the median system. The height of a bar in then given by:
medianp@10(q) medianp@10(q) We can see that our run has 3 queries that achieve the best performance, 17 queries perform better than the median, while 26 queries were worse than the median, and other 5 queries perform in the median line. It means that in a specific task such as medical document retrieval, results are motivating but need improvement.
In terms of the best performance of each query, some queries can achieve a very high result like query 14,22,32,33,39,48 and 49 (more than 70%), while some queries perform near the median like query 2 and 20 (only about 10%).
We show in figure 3, the average recall-precision curve for all 50 topics. There is a tradeoff between recall and precision. We notice that when increasing recall by retrieving more, precision decreases. 4
In our first participation in ehealth information retrieval task at the
CLEFeHealth 2014, we obtained motivating results for the search in a large collection
of medical documents to answer patients’ questions when reading clinical
reports. For future work, we will try to improve the obtained results by including
the notion of semantics between terms of documents [