The 2017 CLEF eHeath Task2 requires to rank the retrieval results given by medical database. The purpose is to reduce e orts that experts devote to nding indeed relevant documents. We utilize a customized Learning-to-Rank model to re-rank the retrieval result. Additionally, we adopt word2vec to represent queries and documents and compute the relevant score by cosine distance. We nd that the combination of the two methods achieves a better performance.
In this task, we rst customize a Learning-to-Rank (L2R) model[
Learning-to-Rank Model
The Learning-to-Rank model has shown good performance [
Query Expansion: In the query expansion stage, we intend to improve retrieval
precision by expanding queries. We apply the similar model proposed in the 2014
TREC Microblog track[
{ Query is submitted to Google and the top-10 concurrent web titles and snippets(if exist) is crawled. { The MeSH database is applied to extract medical terms from titles and snippets.
Feature Extraction: In this stage,we need to extract features of each querydocument pair. When a document is retrieved under a query, it is attached with a weighting score and a rank. So we use the weighting score and the rank from the rst retrieval round as features. To take advantages of di erent retrieval models, we adopt BM25[6], PL2[7] and BB2[8] models to obtain the scores and ranks of the query-document pair. Hence the dimension of the feature vector is six.
Model Training: The L2R model judges the relevance of a query-document pair by using the random forest classi er. We choose the topics and documents of the 2013 and 2014 tasks as the training data. The aforementioned feature vectors are applied to represent query-document pairs in this stage. 2.2
Word2vec Model Assuming a document of n words is D = {d1; d2; :::; dn} , we can represent each word di in D as a vector di. Hence the vector of the whole document vector D can be calculated by the average of vectors di:
D = n1 ∑ di:
1 i n Similarly, a query q could also be represented as a vector q. We can compute the similarity between query q and document D. In this task, we use the cosine distance to compute the similarity between document D and query q: sim(D; q) = cos(D; q) =
D · q ∥D∥ · ∥q∥ : After similarities between the query and documents listed are computed, we can rank these documents in a descend order. 2.3
Combination We use SL(D; q) to denote the score of document D with query q from L2R model, and SW (D; q) to that from Word Vector model. is the weight of SL(D; q) and is the weight of SW (D; q). The nal score is computed as below: S(D; q) = SL(D; q) + SW (D; q); +
Dataset We are provided with development set and test set. In development set there are twenty topics while in test set there are thirty topics. Each topic le contains four parts: { Topic-id { The title of review written by experts { The boolean query manually constructed by experts { The set of PubMED Document Identi ers (PID's) returned by MEDLINE. Since the query of a topic is a boolean query, we remove three words near the negation word "not" to avoid misleading the intension of the query . 3.2
Runs We submit three runs whose descriptions are followed below: run-1: result of the Word Vector model. We use the pre-trained word vectors from Stanford University trained by GloVe model[9]. The size of vocabulary is 2.2M and the dimension of each vector is 300. The vector of the word that does not occur in the pre-trained word vectors is 0. run-2: result of L2R model.We adopt terrier-4.0.0 to run BM25, BB2 and PL2 model. We select top-1000 PIDs for each topic. run-3: result of the combination of L2R model and Word Vector model. The parameters are tuned on training set. Finally we choose = 0:8 and = 0:2 in equation (4). However, a PID of a topic may not occur in the result of L2R model. In this case, = 0 and = 1. Similar to run-2, we choose top-1000 PIDs for each topic.
The evaluation results of three runs are shown in Table.1. These results are provided by the organizer. 4
In the 2017 CLEF eHealth Task 2, we ECNU ICA team take advantages of the Learning-to-Rank model. We also adopt word2vec to represent queries and documents and compute their similarities by cosine distance. Although the combination of two methods performs well, the performance of our word2vec model needs to be improved. In the future, we will apply better methods which can avoid losing semantic information.
Runs run-1 run-2 run-3 Runs run-1 run-2 run-3 Runs run-1 run-2 run-3 Runs run-1 run-2 run-3
This research is funded by the National Nature Science Foundation of China (No. 61602179) and the Science and Technology Commission of Shanghai Municipality (No.15PJ1401700).
This work was supported by Xiaoi Research, by Shanghai Municipal Commission of Economy and Information Under Grant Project No.201602024. 6. Stephen E., Robertson, S.W., Susan J., Micheline H.B., Mike G.: Okapi at TREC3. Proceedings of the Third Text REtrieval Conference (TREC 1994). Gaithersburg, USA 7. Amati, Gianni, Cornelis Joost, and Van Rijsbergen.: Probabilistic models for information retrieval based on divergence from randomness. (2003). 8. Amati, G., Cornelis J., Van R.: Probabilistic models for information retrieval based on divergence from randomness. (2003). 9. Je rey P.,Richard S.,Christopher D.:2014. GloVe: Global Vectors for Word Representation. Empirical Methods in Natural Language Processing (EMNLP) (2014).