Observational evidence in clinical practice is critical in healthcare and policy making. Researchers spend a lot of time searching for relevant published articles to write a systematic review of a topic. In this paper, we present our participation as the team of IIIT Hyderabad at Task2 Technologically Assisted Reviews in Empirical Medicine as an e ort to automate this task and deliver relevant information in medical literature. We base our approach on query expansion according to relevance feedback. Query expansion is a standard technique in information retrieval tasks with growing use in medical literature [1, 2]. Articles returned from pubmed query performed during a systematic review are rst indexed using lucene's inverted index. The query is porcessed for term boosting, fuzzy search and used for scoring documents according to TF-IDF similarity. Relevance feedback is used to update the query and become more pragmatic.
Diagnostic tests are critical to healthcare. Well designed reviews of results from
Diagnostic test accuracy(DTA) studies will help in decision making in medical
domain [
We participate in Task 2: Technologically Assisted Reviews in Empirical
Medicine, evaluating information retrieval of medical documents. The task
focuses on ranking and thresholding methods for DTA reviews. We proposed a
system which is based on query expansion using fuzzy logic and relevance
feedback to get relevant documents. Relevance feedback is used earlier in various
information retrieval systems[6{8]. Fuzzy search make query exible and helps
improve recall. Relevance feedback helps reconstruct the query to deal with any
ambiguous information need [
Each query is initially converted into a fuzzy query. The documents pertaining to each topic are indexed using lucene3. These indexed documents are searched using query provided by Cochrane experts. The query is updated to include more terms from relevant documents from an initial set of ranked documents provided by lucene and remove terms from irrelevant ones. Since the initial ranking of a few documents gives high average precision, the idea is to let unique terms be picked from them to better represent a query. The updated query is used to rank remaining documents. 2
Methodology In this section we explain our methodology in detail. For simple evaluation runs, we try to optimize recall by ranking approximately half of the documents. However, for cost e ective measures, we stop when we don't nd any query updates or average precision in the last set of ranked results falls below a threshold (0.1 in most cases). A summary of the runs submitted to the task is shown in Table 1 .
We let lucene index each topic's documents. Lucene breaks each document into words to create an inverted index. This index consists of terms with set of documents that contain it. Later, it is utilized for e cient search. To reduce noise and false positives, we remove stop words from the documents at the time of indexing. Lucene separates document information into elds. We create elds for title, abstract etc. from pubmed documents, as the queries speci es terms along with elds to search them from. 2.2
The query provided by the Cochrane experts vary in length and have complex boolean logic. We use a fuzzy search system to expand it. The system allows terms close to the base term to be included in the expanded query. For example, search terms like "dysplasia" also include terms like "dysplastic" and "dysplasias". Although the OVID medline search syntax includes some amount of regex present in the query, we make every term go through fuzzy search system before adding it to expanded query. 3 https://lucene.apache.org/core/ After building the document index and query reformulation, we make use of TF-IDF scoring model. Vector space models lets reweigh search terms quickly and uses cosine for calculating similarity between document and query. Four similarity measures are incorporated - tf, idf, coord and length Norm. Where coord is number of terms in the query that were found in the document and length Norm is measure of the importance of a term according to the total number of terms in the eld.
Initially, we request a small set of ranked and scored documents from lucene. This initial set is inspected for relevance. We found from our experiments on the training data that about half of this set is relevant. Let (rd) be the set of relevant document and (nrd) be the set of not relevant documents in the initial ranking. The search query is appended with boolean OR with top occuring terms from rd and boolean NOT from top occuring terms from nrd given that they don't already occur in the query. To prevent overpopulating terms in the query and drifting away from desired result, we restrict the count of new terms ve percent of average article size. Once updated, the new query is used to rank remaining documents.
We boost a term for scoring if it occurs in rd for multiple iterations. These terms get n times as much weight of any other term if they occur again in the nth iteration. Incorporating this, we found that though we are providing a binary relevance feedback, our system has the advantages of a graded feedback. Relevance feedback system is applied to queries containing more than 1500 documents. Apart from the submitted runs, we found that this technique was e ective on queries having less documents. 2.4
We submitted eight runs for this task. Four of which are for simple evaluation and four for cost-based evaluation.