In this paper, we describe the methods deployed in the different runs submitted for our participation to the CLEF eHealth 2017 Task 3: Patient-Centered Information Retrieval, IRTask 1: ad-hoc search. Speci cally, we deploy DPH term weighting model with explicit relevance feedback, where the expansion terms are selected from documents which were previously identi ed as relevant by assessors for each query. As improvement we deployed proximity search using both Full Dependence (FD) and Sequential Dependence (SD) variants of the Markov Random Fields and the Divergence From Randomness (DFR) based dependence models to re-rank documents, which have query terms in close proximity. In another approach, we deploy pseudo relevance feedback, where the expansion terms are selected from the top 3 ranked documents after a rst pass retrieval. In addition, we deploy proximity search using the SD variant of the DFR based dependence model.
In this paper, we describe the methods used for our participation to the CLEF
eHealth 2017 Task 3: Patient-Centered Information Retrieval, IRTask 1:
adhoc search. Detailed task description is available in the overview paper of Task
3 [
In this section, we begin by presenting a brief but essential background on the di erent algorithms used in our experimental investigation and evaluation. We start describing the DPH term weighting model in Section 2.1. We then describe the Bose-Einstein 1 (Bo1) model for query expansion in Section 2.2. 2.1
For all our experimental investigation and evaluation we used the
parameterfree DPH term weighting model from the Divergence from Randomness (DFR)
framework [
In our experimental investiagtion and evaluation, we used the Terrier-4.0
Divergence from Randomness (DFR) Bose-Einstein 1 (Bo1) model to select the
most informative terms from the topmost documents after a rst pass document
ranking. The DFR Bo1 model calculates the information content of a term t in
the top-ranked documents as follows [
+ log2(1 + Pn(t)) 1 + Pn(t)
Pn(t) Pn(t) = tf c N (2) (3) where Pn(t) is the probability of t in the whole collection, tf x is the frequency of the query term in the top x ranked documents, tf c is the frequency of the term t in the collection, and N is the number of documents in the collection.
FAQ Retrieval Platform: For all our experimental evaluation, we used Terrier4.2, an open source Information Retrieval (IR) platform. All the documents (ClueWeb 12 B13) used in this study were rst pre-processed before indexing and this involved tokenising the text and stemming each token using the full Porter stemming algorithm. Stopword removal was enabled and we used Terrier stopword list. The index was created using blocks to save positional information with each term. For pseudo relevance feedback, we used Terrier-4.2 DFR BoseEinstein 1 (Bo1) model for query expansion to select the 10 most informative terms from the top 3 ranked documents. 4
Term Weighting Model: For all our runs, we used the parameter-free DPH Divergence From Randomness term weighting model in Terrier-4.2 IR platform to score and rank the documents in the ClueWeb 12 B13 document collection. ub-botswana IRTask1 run1: We ranked the documents using DPH DFR term weighting. As improvement, we deployed explicit relevance feedback, where we selected expansion terms from the top 3 documents that were explicitly marked relevant by assessors for each query. We used the Terrier-4.2 DFR Bose-Einstein 1 (Bo1) model for query expansion to select the 10 most informative terms from these documents. In addition, we deployed the Full Dependence (FD) variant of the Markov Random Fields for terms dependence. Full Dependence assumes all query terms are in some way dependent on each other. In this work, we experimentally selected a window size of 15, which yielded the highest retrieval performance on the training data. ub-botswana IRTask1 run2: We performed a rst pass retrieval using DPH DFR term weighting model. As improvement, we deployed explicit relevance feedback, where we deployed DFR Bo1 model for query expansion to select the expansion terms. ub-botswana IRTask1 run3: We produced an initial ranking using DPH DFR term weighting. As improvement, we deployed explicit relevance feedback and used the DFR Bo1 model for query expansion to select the expansion terms. In addition, we deployed the Sequential Dependence (SD) variant of the Divergence from Randomness based dependence model. Sequential Dependence only assumes a dependence between neighbouring query terms. In this work, we experimentally selected a window size of 15, which yielded the highest retrieval performance on the training data. ub-botswana IRTask1 run4: We used the parameter-free DPH DFR term weighting model to produce and initial ranking. As improvement, we deployed a simple pseudo-relevance feedback on the local collection. We used the Bo1 model for query expansion to select the expansion terms. We then performed a second pass retrieval on the local collection with the new expanded query. ub-botswana IRTask1 run5: We used ub-botswana IRTask1 run4 as the baseline system. As improvement, we deployed the Sequential Dependence (SD) variant of the Divergence from Randomness based term dependence model. Sequential Dependence only assumes a dependence between neighbouring query terms. In this work, we experimentally selected a window size of 15, which yielded the highest retrieval performance on the training data. 5
These working notes were compiled and submitted before the relevance judgments were released. Below we present the results of our runs using the 2016 query relevance judgments. Please note that the o cial results to be released will be di erent because new query relevance judgments will be released Table 1 presents our results on the training data. From this table, we see a degradation in performance when we incorporate term dependence only in our ranking (ub-botswana IRTask1 run5 ). However, when we deploy pseudo relevance feedback (ub-botswana IRTask1 run4 ), we see an improvement in the retrieval performance in terms of precision at 5 (P@5), precision at 10 (P@10) and recall (rel ret). Moreover signi cant improvement in the recall is obtained when explicit relevance feedback is deployed ((ub-botswana IRTask1 run1 ), (ubbotswana IRTask1 run2 ) and (ub-botswana IRTask1 run3 )). In addition, we obtain mixed results when we incorporate proximity search after deploying explicit relevance feedback. For example, there was an improvement in the retrieval performance in terms of P@5 and P@10 when we deploy the FD variant of the Markov Random Fields for term dependence using a window size of 15 (ubbotswana IRTask1 run1 ). In contrast, we obtain a degradation in the retrieval performance in terms of P@5 and P@10 when we deploy the SD variant of the Divergence from Randomness based term dependence model using a window size of 15 (ub-botswana IRTask1 run5 ).