Retrieving relevant information from biomedical text data is a new challenging area of research. Thousands of articles are being added into biomedical literature each year and this large collection of publications ofer an excellent opportunity for discovering hidden biomedical knowledge by applying information retrieval (IR) and Natural Language Processing (NLP) technologies. Biomedical Text processing is diferent from others. It requires special kind of processing as it has complex medical terminologies. Medical entity identification and normalization itself is a research problem. Relationships among medical entities have the impact on any system. The Clinical Decision Support systems are aimed to provide assistance to the decision-making tasks in biomedical domain. The medical knowledge have the potential to impact considerably on the quality of care provided by clinicians. Medical field has various types of queries: short questions, medical case reports, medical case narratives, verbose medical queries, community questioning, semi-structured queries, etc. These diverse nature of medical data demands special kind of attention from IR and NLP.
• Information systems → Query reformulation; Document ifltering ; Clustering and classification ;
MOTIVATION AND CHALLENGES The recent statistics shows that 70% of total web search queries are of medical and healthcare category. Biomedical Information Retrieval(BIR) is a special type of information retrieval. Major challenges in biomedical information retrieval are in handling complex, ambiguous, inconsistent medical terms and their ad-hoc abbreviations.
• There are many complex terms like ’nuclear factor kappalight-chain-enhancer of activated B cells’, ’NF-kB DNA binding with electromobility shift assay’. The average length of biomedical entities is much higher than general entities.
Identifying such medical entities is a preliminary subtask. • Physicians use ad-hoc abbreviations very frequently and they are ambiguous like ’PSA’ can be ’prostate specific antigen’ or ’psoriasis arthritis’ or ’poultry science association’. • The rapid change in terminologies makes them inconsistent. For instance ’H1N1 influenza’, ’H1N1 Virus’, ’swine influenza’, ’SI’, ’Pig Flu’ and ’Swine-Origin Influenza A H1N1 Virus’, all refers to the same entity. Such diferent diferent representations of the same entity should be normalized to a single representation. This problem is known as entity normalization. This type of problems of acronym disambiguation leads to poor system performance.
Also, there can be two types of users of the healthcare related search systems : experts(clinicians) and laymen(other than clinicians). The query formulations of both the users are diferent for the same information need. For example, general people use the words ’heart attack’, ’Irregular heartbeat’, ’Mouth ulcer’ while medical practitioners/experts use the words ’Myocardial infarction’, ’Cardiac arrhythmia’, ’Mucosal ulcer’ respectively. This leads to the problem of vocabulary mismatch where diferent people name the same thing or concept diferently. As an efect of the characteristics of biomedical terminologies and user dependent query formulations, the problem of vocabulary mismatch between query and documents(relevant) arises in Biomedical Information Retrieval. Missing synonyms causes low retrieval recall i.e. out of all relevant documents in the collection, very few relevant documents get retrieved. Also, ambiguous terms cause low precision i.e. out of all retrieved documents, very few are relevant. We need to construct such Biomedical search engines that can address the above issues. 2
Widely-used text collections in the biomedical domain are MEDLINE/PubMed, OHSUMED, and GENIA.
• The MEDLINE/PubMed database contains bibliographic
references to journal articles in the life sciences with a
concentration on biomedicine, and it is maintained by the U. S.
National Library of Medicine (NLM). This MEDLINE/PubMed
records can be downloaded for research.
• The OHSUMED [
MEDLINE citations (1994-2003).
• The TREC Clinical Decision Support data [
LITERATURE SURVEY
’Information Retrieval: A Health and Biomedical Perspective’ [
PubMed is a biomedical search engine which accesses primarily
the MEDLINE database of abstracts and references on biomedical
topics and life sciences and is maintained by the United States
National Library of Medicine (NLM)1 at the National Institutes of
Health (NIH). PubMed does binary matching[
On the contrary medical and healthcare related queries are longer than general queries since people used to describe the symptoms, tests and ongoing treatments. For verbose and longer queries, biomedical IR systems should deal properly with ambiguous, complex and inconsistent biomedical terminologies which is dificult to handle.
Automatic processing of biomedical text sufers from lexical
ambiguity (homonymy and polysemy) and synonymy. Automatic
query expansion (AQE) [
The emergence of medical domain specific knowledge like UMLS
can contribute to the retrieval system to gain more understanding
of the biomedical documents and queries. Various approaches of
information retrieval with the UMLS Metathesaurus have been
reported: some with decline in results[
BIOMEDICAL DOCUMENT RETRIEVAL
Preliminary Experiments
Query Expansion which uses the top retrieved relevant documents
is known as Relevance Feedback since it uses the human judgement
to identify the relevancy. Pseudo Relevance Feedback technique
assumes the top retrieved documents relevant and uses as feedback
documents. The Query expansion based approches for biomedical
domain gives better results [
Table 1 shows the results of standard retrieval, Pseudo-Relevance
Feedback (PRF) based Query Expansion and Relevance Feedback
(RF) based Query Expansion with BM25 [
Feedback Documents Discovery
Query expansion method largely rely on feedback documents and
feedback terms. Automatic query expansion methods based on
pseudo relevance feedback uses top retrieved documents as
feedback documents.[
One attempt is to learn the truly relevant documents for feedback by using minimum human intervention. The approach uses human judgements for a small set of feedback documents and then it tries to learn identifying true relevant documents from rest of the documents. Then the documents identified relevant are used for feedback and query expansion is performed. Two approaches for this learning based on classification and clustering are presented here.
First Algorithm: The first proposed algorithm is based on classification. If we have human judgements available for some of the feedback documents, then it will serve as a training data for classification. The documents are represented as a collection of bag-of-words, the TF-IDF scores of the words represent features and human relevance scores provides the classes. By using this as a training data, we want to predict the relevance of other top retrieved feedback documents.
For each query Q (1) DN - set of N top retrieved documents {d1, d2, ..., dN } (2) Dk - set of k top retrieved documents for which human judgements are available {d1, d2, ..., dk } (3) Dl - set of l=N-k top retrieved documents for which human judgements are not available {dk+1, dk+2, ..., dN } (4) DF - set of feedback documents (5) DF = {di ; relevance o f di > 0, di ∈ Dk } (6) Train a classifier C on Dk using relevance as a class label and generate model Mc (7) For each document dj in Dl , k + 1 ≤ j ≤ N (8) Predict the relevance rj of dj using trained model (9)
Mc
If rj > 0, then DF = DF ∪ {dj } and it is responsible for insignificant improvement. This approach further removes non-relevant documents from relevant document class identified by classification approach. The idea is to perform clustering on the relevant identified documents with number of clusters two: one from actually relevant documents and second from non-relevant documents. K-means clustering is used with k=2. Since, the convergence of K-means clustering depends on the initial choice of cluster centroids, the initial cluster centroids are choosen as the average of relevant documents’ vectors and the average of non-relevant documents’ vectors from training data.
For each query Q (1) DN - set of N top retrieved documents {d1, d2, ..., dN } (2) Dk - set of k top retrieved documents for which human judgements are available {d1, d2, ..., dk } (3) Dl - set of l=N-k top retrieved documents for which human judgements are not available {dk+1, dk+2, ..., dN } (4) DF - set of feedback documents (5) DF = {di ; relevance o f di > 0, di ∈ Dk } (6) Train a classifier C on Dk using relevance as a class label and generate model Mc (7) DR = ϕ, DN R = ϕ (8) For each document dj in Dl , k + 1 ≤ j ≤ N (9) Predict the relevance rj of dj using trained model (10) (11) The query expansion considers top N retrieved documents for feedback. Here, we have considered top 250 documents, from which the set of top 50 documents are used as training i.e. human judgements for top 50 documents are used in training and rest of 200 documents are taken for testing data. The relevance is predicted for those 200 documents and only relevant predicted documents are then used for feedback. The result of relevance feedback using top 50 documents is the baseline for other results. All the computed results are compared with the baseline.
The experiments are performed using nine diferent classifiers for classification in first algorithm. The table 2 shows the results in terms of MAP score for CDS 2014 dataset. Neural-Net gives best result among all nine classifiers. Also, the result of classification with Nearest-Neighbors is comparable to the baseline.
The classification results are not significant to the baseline results. We investigated the matter and came to know that the relevant classified documents in relevance class are not all actually relevant. The feedback document set also contains some irrelevant documents (misclassification). For all the 30 queries of CDS 2014, classification Nearest-Neighbours classified 625 documents as relevant out of all 200*30 documents. Out of 625 documents used for feedback, 244 documents were actually relevant while other 381 documents were wrongly classified as relevant. So, these 381 irrelevant documents are noise to the system. The second approach takes this matter into consideration and further refine the feedback document set by performing 2-cluster clustering on 625 documents. Manually removing 381 irrelevant documents from feedback document set shows significant improvement over baseline. The results of manually removing false classified documents from feedback set and automatic clustering approach are also shown in the table 2.
The same experiments are performed on CDS 2015 and 2016 datasets. The results of both the algorithms using six diferent classifiers are shown in tabel 3. For CDS 2015 dataset second algorithm performs better than baseline but the diference is not significant. For CDS 2016 dataset, both the algorithms perform similar to the baseline.
MAP
Gaussian-Process Decision-Tree Random-Forest Neural-Net AdaBoost Naive-Bayes 0.2768 0.2761 0.2736 0.2736 0.2736 0.2496 0.2733 0.2790 0.2618 0.2614 MAP
Nearest-Neighbors Decision-Tree Random-Forest Neural-Net AdaBoost Naive-Bayes 0.2283 0.2234 0.2065 0.2130 0.2295 0.2092 0.2172 classification classification + manually removing false Relevant docs classification + clustering 0.2768 0.2815 (p = 0.048) 0.2760 0.2760 0.2762 0.2788 0.2760 0.2808 0.2806 0.2792 0.2768 0.2794 (p = 0.305) 0.2750 0.2750 0.2753 0.2725 0.2747 0.2790 0.2741 0.2661 CDS 2016 classification classification + clustering classification classification + clustering 0.2283 0.2324 (p = 0.115) 0.2218 0.2281 0.2299 0.2213 0.2269 0.1456 0.1456 0.1138 0.1450 0.1460 0.1255 0.1436
0.1456 0.1459 (p = 0.895) 0.1370 0.1458 0.1466 0.1345 0.1468