UMass at BioASQ 2014: Figure-inspired Text Retrieval Jesse Lingeman and Laura Dietz School of Computer Science, University of Massachusetts, Amherst {lingeman,dietz}@cs.umass.edu Abstract. Building on our experience with retrieval of gures, gure summarization with sentences from text, we study the utility of gure- based features and techniques for text retrieval. Figure based approaches are compared to approaches using abstracts instead of gures. We also explore two dierent relevance models: one built using the Unied Med- ical Language System (UMLS) and one built using Wikipedia. We conduct several experiments exploring dierent feature combinations using a model built with the TREC Genomics track for submission to the 2014 BioASQ competition. 1 Introduction The BioASQ competition is about answering biomedical questions by extracting information from research publications on Pubmed. BioASQ oers several sub- tasks to participate in: retrieving Pubmed documents that contain an answer, retrieving snippets from those documents that contain an answer, retrieving relevant concepts or RDF triples, and extracting the answer from all retrieved material. In a cooperation between the Center for Intelligent Information Retrieval and UMass Amherst and the BioNLP group at UMass Medical school in Worcester, we developed a gure-inspired text retrieval method as a new way of retrieving documents and text passages from biomedical publictions. Our method is based on the insight that for biomedical publications, the gures play a central role up to the point where their caption and references provide abstract-like summaries of the paper. In this work we build on our experience with gure summarization and gure ranking algorithms [5,8,1]. We are test driving our gure-inspired retrieval method in the BioASQ com- petition, where we focus our participation on document and snippet retrieval. As gures are the center of our attention, our methods rely on the availabil- ity of full text, e.g. in PMC format. Therefore we only retrieve documents and snippets contained in Pubmed Central. We notice that the available training data covers Pubmed Central only sparsely. Most queries in the gold standard contain just one publication from Pubmed Central; only 13 queries contained at least 10 documents in Pubmed Central. Since it is infeasible to dene a complete gold standard ahead of time, our mission is to identify new material from PMC 1296 Table 1. Examples of relevant snippets in PMC. 5319ac18b166e2b806000030 Is clathrin involved in E-cadherin endocytosis? plasma membranes we have found here that non-trans-interacting e-cadherin is constitutively endocytosed like integrin ligand-independent endocytosis that the formation of endocytosed vesicles of e-cadherin is clathrin dependent and that e-cadherin but not other cams at ajs and tjs including nectins claudins and occludin is selectively sorted into the endocytosed (PMC 15263019) 5319abc9b166e2b80600002d Is Rac1 involved in cancer cell invasion? cells was clearly demonstrated by rna interference assay rac1 depletion signicantly suppressed the frequency of invasion in both quiescent and igf-i-stimulated mda-mb-231 cells this indicates the necessity of rac1 for igf-i-induced cell invasion in the cells overexpression of rac1 has been (PMC 21961005) that answers the questions. To demonstrate the existence of relevant material we show examples of relevant snippets in Table 1 and provide more examples in the result section. In the absence of suitable training data on full documents, we develop and train our method on data from TREC Genomics track 2006 and 2007. Like Bioasq Task 2b(phase A), the Genomics TREC task focuses on retrieving rel- evant documents and snippets for biomedical questions. The distinctions lie in the use of the Highwire corpus. After training supervised models on the TREC data, they are applied to questions posed in the BioASQ competition. Our approach takes an Information Retrieval perspective on the problem. First, query expansion is performed with information from UMLS, Wikipedia, and Figures to enrich the question. Second, a ranking of full documents and snippets is retrieved from a corpus of articles from Pubmed Central. Third, we extract features for each document and snippet that indicate its relevance for the question and re-rank document/snippets with a supervised learning-to-rank approach. 2 Background: Information Retrieval This section introduces document retrieval models and query expansion tech- niques. 2.1 Sequential Dependence Model An early IR method called query likelihood employed an independence as- sumption within query terms to score documents with Dirichlet collection smooth- ing. For query terms q1 , q2 , ...qm , each document D in the collection is scored by a product of scores under each query term. m #(q , D) + µ #(qi ,·) Y i #(·,·) scoreuni (q1 ,q2 ,...qm ) (D) = log (1) i=1 #(·, D) + µ 1297 We use the notation '·' to denote sums over all possible entries. In particular #(qi , D) refers to the term frequency of qi in the given document, #(qi , ·) refers to the term frequency of qi in the corpus, and #(·, D) is the document length and #(·, ·) number of terms in the collection. The scalar µ controls the amount of collection smoothing applied, and is a hyperparameter to be estimated. Good values of µ are in the range of [500, 5000]. The query likelihood model is almost always outperformed by the sequen- tial dependence model [6], which also includes exact bigrams and windowed skip-bigrams. The unigram model above can be generalized to arbitrary count statistics, such as occurrences of a bigram ”qi qi+1 ” in document D to derive scorebi . Furthermore, counting co-occurrences of the two terms qi and qi+1 in any order within a window of 8 terms in the document gives rise to the score under the windowed bigram model score , where the marginal counts in the wbi denominator #(·, D) are approximated by the document length. The sequential dependence model combines the scores of the document D under the unigram, bigram and window model as a log-linear model. scoreSDM (q1 ,q2 ,...qm ) (D) = λuni scoreuni (D) + λbi scorebi (D) + λwbi scorewbi (D) = < λ, φ(D) > (2) The sequential dependence model requires setting of hyperparameters λuni ,λbi , λwbi , and µ, where the λs can be estimated with machine learning. 2.2 Query Expansion Keyword-based retrieval methods such as query likelihood and sequential de- pendence fail to retrieve documents that refer to the query terms via synonyms. A solution is to expand the original query q1 , q2 , ...qm with additional terms t1 , t2 , ...tK so-called expansion terms. Methods for predicting expansion terms ti also provide condence weights wi . An expanded SDM query scores documents D by X scoreQ (D) = scoreSDM (q1 ,q2 ,...qm ) (D) + ω · wi · scoreuni (ti ) (D) (3) i The expanded retrieval model introduces another hyperparameter ω , which can be estimated along with λ using machine learning. 2.3 Pseudo-relevance Feedback Additional expansion terms can be derived from external synonym resources or estimated with pseudo-relevance feedback. In pseudo relevance feedback the expansion terms are estimated from the document collection [3]. The approach is based on the assumption that the un-expanded retrieval model obtained high precision in the top ranks, but was lacking recall. 1298 The procedure gathers a feedback ranking D1 , D2 , ..., Dn from the documents from the collection which have the highest score under the un-expanded query, e.g. score (D). SDM The next step derives distribution over terms from the feedback documents. This involves taking the score of the document Di to approximate a relative retrieval probability of Di compared to the rest of the feedback set. 1 p(Di |q1 , ..., qm ) = Pn exp scoreSDM (Di ) (4) j=1 exp scoreSDM (Dj ) In addition, for each feedback document, a distribution over terms is derived as a language model. #(t, Di ) p(t|Di ) ∝ (5) #(·, Di ) These two parts are aggregated to estimate the term distribution for expan- sion. We derive the estimator as a mixture of document-specic language models where the document retrieval probabilitie govern the mixing weights. n X p(t) = p(t|Di )p(Di |q1 , ..., qm ) (6) i=1 The K most probable terms ti under this distribution, together with weights w = p(ti ) are predicted as expansion terms. 2.4 Learning Hyperparameters We exploit that a SDM retrieval model with query expansion falls into the family of log-linear models which can be eciently estimated with a learning-to-rank approach [7]. We represent each document by a feature vector with four entries: the document's score under the unigram model, as well as the bigram, window- bigram, and expansion model. We use the document relevance assessments from the training set to estimate a log-linear learning-to-rank model. In this work we use the coordinate ascent learner from the RankLib 1 package optimizing for the metric mean-average precision (MAP). The weights of the optimal learning-to-rank model are also the optimal set- tings λuni ,λbi , λwbi and ω for the retrieval model. When the SDM model is ex- panded with multiple expansion models this learning-to-rank approach can be generalized appropriately. This reduces the hyperparameters that need to be estimated by grid-tuning to the Dirichlet smoothing µ for SDM, and number of feedback document n and number of expansion terms K for each expansion model. 1 http://people.cs.umass.edu/~vdang/ranklib.html 1299 3 Retrieval Approaches In this section we detail how retrieval and query expansion approaches are com- bined to leverage gure information to derive a rst pass of bio-medical text retrieval. We discuss reranking techniques in Section 4. We refer to the target document collection as full documents, as we further extract pseudo-documents for gures and abstract. 3.1 Indexes From the full documents in the collection, we create dierent retrieval indexes. The full document index contains the documents in Pubmed Central document collection. The task is to retrieve relevant documents from this col- lection. The collection is converted into JSON format using the convertion tool provided by the BioASQ organizers. We index the all visible text as-is while pre- serving character osets and section information. The document preprocessing uses a special tokenizer that preserves the names of chemical compounds, genes and pathways. We identify all gures in the original Pubmed central format and extract so- called figure documents for each of them. The gure document includes the caption of the gure, the sentences that reference the gure. In separate elds we also include sentences within a window of one and two sentences away from a gure reference. We use the gure documents for query expansion and feature generation. In order to compare the expressiveness of gure documents to abstracts, we also create an index of abstracts that we swap in as a replacement for gure documents. 3.2 Document Retrieval The most basic retrieval method uses the given query Q to obtain a ranking of full documents under the sequential dependence model. This ranking can be output directly [UMass-irSDM], or submitted to a feature-based re-ranking method (described in Section 4). We can improve the ranking by expanding the original query with expansion 0 terms (to obtain query Q ) to derive a ranking the full documents. To expand the query with pseudo-relevance feedback, we have dierent options. We can employ the gure document index [FigDoc Query Expansion] to retrieve a feedback run, compute term distributions according to the relevance model and expand the query Q. This approach is also applied to the index of abstract documents to derive the method [Abstract Query Expansion]. As an external source of synonyms we can also use Wikipedia. For that we create a full text index of a Wikipedia snapshot from January 2012 which contains articles for dierent entities, where some are targeting the biomedical domain. We cast the original query to our Wikipedia index and apply standard pseudo-relevance feedback [Wiki Query Expansion]. 1300 Alternatively, we expand the query using an external synonym dictionary. In this study we use the Unied Medical Language System (UMLS) [4,2]. We look up all query terms qi and all query bi-grams qi qi+1 in the UMLS dictionary to build a pool of expansion terms. Prioritizing for terms that are returned by more than one lookup, we identify K expansion terms [UMLS Query Expansion]. In all approaches we learn the SDM parameters λ and expansion weight ω using 25% of the TREC Genomics queries as training data. We tune the hyperpa- rameter µ of the sequential dependence model using grid-tuning on another 25% of the TREC queries as validation data. We select the maximal µ and according λ and ω and keep it xed for the remainder of the experiment. 3.3 Snippet Retrieval To participate in the snippet retrieval task, the goal is to break down the relevant documents into passages that are likely to contain the answer. In the eld of Information Retrieval this problem is known under the name Answer-Passage Retrieval. The passage retrieval approach applies the document retrieval model to con- secutive text segments inside the document, to create a ranking on the sub- document level. We chose a granularity of 50 words, which are shifted through the document in increments of 25 words. For eciency reasons we only consider documents in the high ranks for passage retrieval. For each document, we only consider the highest ranking passage (called Max-Passage) in the following. 4 Feature-based Re-ranking Approaches The ranking of full documents created by methods in Section 3 can be further improved with a supervised re-ranking approach. We use four main classes of features. IR Features (Table 2) are derived from the retrieval score under the unigram, bigram, windowed bigram, and expansion model. The Fiat Docu- ment Features (Table 3) are based on similarity measures between the query and a semi-structured representation of the full document. Figure captions are included in the text, but not regarded in any special way. The Fiat Figure Fea- tures (Table 4) are designed to capture similarity of the query to gure-related information available in the semi-structured document. The fourth category are Figure Document Features (Table 5) which are derived by retrieving gure documents (or abstracts), generate features for every gure, and aggregating across gures within the same document. A full list of features can be found in the appendix. The main idea behind the gure and gure document features is to use gures as a way to easily isolate important text. There is a lot of technical content in ar- ticles, such as related work sections or details on the experimental setup, that are not necessarily relevant to the question being asked and can skew search results. Figures and gure-related passages, on the other hand, are usually describing 1301 Table 2. IR Features for Reranking Feature Name Type Description docscore IR Overall score of the document docrank IR Overall rank of the document docexpscore IR Exponentiated score of the document docrecrank IR Reciprocal rank of the document unidocscore IR Unigram model score unidocrecrank IR Unigram model rank unidocexpscore IR Unigram model exponentiated score unidocrecrank IR Unigram model reciprocal rank bidocscore IR Bigram model score bidocrank IR Bigram model rank bidocexpscore IR Bigram model exponentiated score bidocrecrank IR Bigram model reciprocal rank wbidocscore IR Windowed bigram model score wbidocrank IR Windowed bigram model rank wbidocexpscore IR Windowed bigram exponentiated score wbidocrecrank IR Windowed bigram reciprocal rank expdocscore IR Expansion model score expdocrank IR Expansion model rank expdocexpscore IR Exponentiated score of expansion model expdocrecrank IR Reciprocal rank of expansion model maxpsgscore IR Maximum passage score in the document maxpsgrank IR Highest rank of passage in document maxpsgexpscore IR Exponentiated maximum passage score maxpsgrecrank IR Reciprocal of highest ranked passage Table 3. Document Features for Reranking Feature Name Type Description abs.in_abstract Passage Is passage in abstract? tbl.tdf Passage TF-IDF between passage and table captions tbl.query_cover Passage Query cover (QC) of referenced table captions tbl.num_refs Passage Number of references to tables in passage cite.tdf Passage TF-IDF between passage and cite.query_cover Passage QC of sentences with references to citations cite.num_refs Passage Number of citations in passage allrefs.tdf Passage TF-IDF to text with refs to gures, tables, or citations allrefs.query_cover Passage QC of references in passage to gures, tables, or citations allrefs.num_refs Passage Number of references in this passage title.tdf Document TF-IDF between the query and the title title.query_cover Document QC of document title abs.tdf Document TF-IDF between the query and abstract abs.query_cover Document QC of the abstract of the document fulltxt.tdf Document TF-IDF between query and the full text of the document fulltxt.query_cover Document QC of the full text of the document Table 4. Figure-Specic Features for Reranking Feature Name Type Description g.num_refs Passage Number of references to gures in passage g.query_cover Passage QC all gure-related sentences referenced in psg g.query_cover_caption Passage QC of gure captions referenced in this passage g.tdf Passage TF-IDF to gure related sentences referenced in psg g.tdf_caption Passage TF-IDF between query and referenced gure captions g.psg_caption_overlap Passage Overlap between passage and referenced gure caption g.in_caption Passage Is this passage inside of a gure caption? g.refs.query_cover Document QC of gure-related sentences g.refs.query_cover_window1 Document QC 1 sentence window around gure-related sentences g.refs.query_cover_window2 Document QC 2 sentence window around gure-related sentences g.refs.tdf Document TF-IDF of gure-related sentences g.refs.tdf_window1 Document TF-IDF 1 sentence window around gure-related sents g.refs.tdf_window2 Document TF-IDF 2 sentence window around gure-related sents g.cap.query_cover Document QC of gure captions in document g.cap.tdf Document TF-IDF between query and all gure captions in doc g.refs.has_gs Document Does this document have gures? g.refs.num_gs Document Number of gures in document 1302 Table 5. Figure Document Features for Reranking Feature Name Type Description gdoc.avgscore FigDoc Average score of gure documents for a given document gdoc.avgrank FigDoc Average rank of gure documents for a given document gdoc.gcount FigDoc Total number of gure document returned gdoc.gcount1 FigDoc Number of gure documents returned at rank 1 gdoc.gcount3 FigDoc Number of gure documents returned at rank 3 gdoc.gcount5 FigDoc Number of gure documents returned at rank 5 gdoc.gcount10 FigDoc Number of gure documents returned at rank 10 gdoc.gcount20 FigDoc Number of gure documents returned at rank 20 gdoc.gcount50 FigDoc Number of gure documents returned at rank 50 gdoc.gcount100 FigDoc Number of gure documents returned at rank 100 gdoc.gcount1000 FigDoc Number of gure documents returned at rank 1000 gdoc.maxscore FigDoc Maximum score of returned gure documents gdoc.minrank FigDoc Minimum rank of returned gure documents gdoc.avgreciprank FigDoc Average reciprocal rank of returned gure documents gdoc.maxreciprank FigDoc Maximum reciprocal rank of returned gure documents an important nding of the article. Here, we use the index of gure documents to extract features capturing the essence of ndings. The query is issues against the FigDoc index and we keep track how many and at which rank we retrieve gures for the respective document. We also keep track whether high ranking gures are referenced from the highest scoring passage, and measure the textual similarity between passage and high ranked captions. This allows to separate the false positives from the true positives: an article may be highly ranked because of something discussed in the related work or future work sections, however an article that may be slightly lower ranked but has relevant gure documents may be the more relevant document. We also use features considering the document as a whole. We generate binary values for quality indicators, e.g., whether a document has gures, citations, and tables. We also generate features about the passages, such as number of gure references, number of citation references, number of table references, and the sum of all references in a passage. Binary features are also calculated for whether or not a passage is in a gure caption or in a document abstract. Most of the generated features compare the tokens in the query to the tokens of some part of the document. Two measures are used to do this: Query Cover and TF-IDF. Query Cover is a simple proportion of how many of the query tokens appear in a particular part of the document. TF-IDF is similar, but each token is weighted by how frequent it appears in the corpus. If a token does not frequently appear in the corpus, but appears often in a part of the document, it gets a higher score than if it is a common token in the corpus. These measures are evaluated over dierent segments of the document: we obtain scores by comparing the query to the document abstracts, sentences in the document that reference a gure, a window of sentences around a gure reference, gure captions, and sentences in the document that reference a citation or table. 5 Experimental Evaluation We train and validate our methods on test sets of the TREC Genomics track from the years 2006 and 2007. Both test sets make use of a collection of 162,259 1303 Table 6. Overview of dierent methods used in the TREC Genomics evaluation. Rerank FigDoc Rerank Doc Rerank Fig Rerank All All no RM Rerank IR IR SDM IR RM FigDoc Query Expansion X X X X X X IR Full Docs X X X X X X X X IR Figure Docs X X X Supervised Re-ranking X X X X X X Features IR Doc / Passage X X X X X X Features Full Docs (Text Only) X X X X Features Figures from Full Docs X X X Features Figure Document X X X documents from 59 biomedical journals published by Highwire Press. The docu- ments are made available as raw HTML with several download errors and partial documents. The 2006 collection comprises 27 queries and the 2007 collection in- clude 35 queries. In the following, we make use of a development set comprising the union of the rst half of queries from both 2006 and 2007 test collections for feature devel- opment and hyperparameter tuning. We report results on both the development set and the combined test sets from 2006 and 2007. 0.45 0.40 0.35 0.30 DOCUMENT 0.25 0.20 0.15 0.10 0.05 0.00 ll DM M IR oc k Fi g oc A M IR S IR R ank kD an igD ank oR Rer Rer an Rer ran kF Rer All n Re Fig. 1. Cross-validation results on TREC Genomics development set in mean-average precision (MAP). 1304 5.1 Retrieval Hyperparameters Settings of hyperparameters for retrieval models are determined on the BioASQ training data, which we further subdivide into a 50% training-fold for log-linand a 50% validation-fold. We train the sequential dependence parameters λuni ,λbi , λwbi and relevance model balance-weight ω in log-linear model fashion with coordinate ascent (using the RankLib package) on the training fold. We tune the Dirichlet smoothing parameter µ on a selection of 100, 1000, 2000, 2500, 3000 on the validation fold. The parameter settings change with the system. As we aggregate more BioASQ training data from the previous batch submissions (query for task 2b phase b), the parameters also change across batches. A detailed list of which parameter has been used in which batch is given in Table 9. 5.2 Retrieval and Reranking Methods We study the impact of dierent components on the overall document retrieval eectiveness, by omitting some components from the pipeline as indicated in Table 7. The most complete method, referred to as All-Figdoc-UMLS includes all elements of our pipeline: query expansion on the Figure Document index, retrieval of full documents with the expanded query, generation of various fea- tures for re-ranking. The feature sets include scores from the IR system as well as text-only features in addition to gure-related features as extracted from the full documents and Figure Documents. 5.3 Training Supervised Re-ranking on TREC Genomics As only few BioASQ training queries have more than 10 positive documents in the Pubmed Central collection, we were hesitant to train the supervised re- ranking model on it. We learn the parameter vector for feature-based reranking on the TREC Genomics queries test set, using years 2006 and 2007 on the corpus of Highwire publications. We use 50% of the TREC queries for learning the supervision. As the supervision depends on IR hyperparameters, we apply the tuning heuristic above to 25% of the TREC queries (yielding λuni = 0.77, λbi = 0.005, λwbi = 0.037, ω = 0.20 and µ = 2500). 5.4 Evaluation on TREC Genomics We study dierent components of our methods on TREC Genomics holdout set. We evaluate the Rerank All method (corresponding to system All-Figdoc- UMLS) method compared to variants of this approach that omit certain feature classes or steps in the retrieval pipeline. An overview of the evaluated methods is given in Table 6. The ocial evaluation metric of the TREC Genomics test set is mean-average precision (MAP) on the document ranking. The results on the development set are presented in Figure 1. We see that the re-ranking approaches gain a decent 1305 boost, whereas the dierences between dierent feature sets are neglegible. With a paired-t-test at signicance level α = 5%, we verify that Rerank All and Rerank Doc yield signicant improvements over both IR baselines (despite the overlap in error bars). 5.5 Submission to BioASQ We restrict all rankings to the top 20 documents, and for each document we provide the best scoring snippet, yielding 20 snippets per system and query. We score snippets with the same retrieval model that we use for document retrieval. Inspecting all top 50 documents, for each document we create snippet candi- dates by a sliding window of 50 terms (shifted by 25 terms) and only return the snippet with the highest score under the expanded retrieval model. The snippets are reranked by the retrieval score under the passage model and we only out- put the top 20 snippets. This means, that some snippets might stem from new documents. The term windows are converted to section IDs and character osets. In the batch 1 submission, we did not incorporate whitespaces and XML formatting correctly. This has been corrected for all remaining batches. Table 7. Overview of dierent systems submitted to the BioASQ evaluation. 'X' de- notes that the component was selected in all batches for this system. Components only selected in some batches are indicated with 'B'. All-Abstract-UMLS Doc-Figdoc-UMLS All-Figdoc-UMLS UMass-irSDM All-Figdoc FigDoc Query Expansion X X X Abtract Query Expansion X UMLS Expansion B1, B2 B1, B2 B1, B2 B1-B5 Wikipedia Expansion B3, B4, B5 B3, B4, B5 B3, B4, B5 IR Full Docs X X X X X IR Figure Docs X X X X Supervised Re-ranking X X X X Features IR Doc / Passage X X X X Features Full Docs (Text Only) X X X Features Figures from Full Docs X X X Features Figure Document X X X X We modied the some components across dierent submitted batches, to maximize our knowledge gain in the light of the limitation to 5 submission sys- 1306 tems. In particular we varied the query expansion with external sources, from using UMLS to Wikipedia. This change is indicated in Table 7. Timing. The methods were run on a gridengine cluster each node having a 2.21GHz Intel Xeon CPU with 10GB of RAM (much more than necessary). Aver- aging the CPU time of 100 queries, we observe 21 seconds for irSDM, 35 seconds for All-FigDoc-UMLS (with Wikipedia Expansion), 41 seconds All-Abstract- UMLS, 25 seconds for All-FigDoc, 36 seconds for Doc-Figdoc-UMLS. Results. After observing an abysmal score for all our systems on the ocial preliminary results, we manually inspected the quality of predicted snippets on rank one and two in 25 queries of batch 5 obtained by the irSDM method. Table 10 displays some of the relevant snippets. We notice that many of the documents are not listed in the gold standard. An exception are the query on archeal genomes where we found a much more descriptive snippet than the one provided in the gold standard, and the query on Gray paleted syndrome, where our passage includes the ground truth passage. We perform a more elaborate annotation on a subset of nine queries from batch 3 (irSDM). The results, measured in snippet precision at rank 10 (P@10) are presented in Table 8. We see that the precision varies between 10% and 70%, but all queries have a non-zero precision. One of our common mistakes occurs when questions ask about a particular brand of medicine or active ingredient. We notice that in such cases, a large percentage of retrieved snippets are about the disease in general, but do not mention the brand or ingredient. In the future, we intend to modify our approach by identifying such required words with an NLP tagger such as conditional random elds and discard snippets that do not contain the required word. Table 8. P@10 of snippets returned by irSDM on nine selected queries. Query P@10 52b2efcb4003448f55000005 0.1 52b2e97df828ad283c000012 0.2 52b2ed144003448f55000004 0.3 52b2ec944003448f55000002 0.6 52b06a68f828ad283c000005 0.7 52b2e409f828ad283c00000e 0.4 52b2ecd34003448f55000003 0.1 52b2e1d8f828ad283c00000c 0.2 52b2f09f4003448f55000008 0.2 average 0.3 1307 6 Conclusion For the UMass BioASQ submission we designed a gure-aware IR system which includes search-indexes of full document as well as gure captions and refer- ences. We use gures both as a resource for query expansion and test external source such as Wikipedia and UMLS as well. The retrieval approach is comple- mented by a supervised learning-to-rank method the includes features from IR, the document, gure features, and features from retrieving gure documents. We evaluate against a very strong text-only baseline, which is outperformed on our development test set from the TREC Genomics track. We anticipate that including features from the gure-documents in both the retrieval methods and in reranking will improve the ranking of both document and snippets. Acknowledgements This work was supported in part by the Center for Intelligent Information Re- trieval, in part by Umass Medical School subaward RFS2014051 under National Institutes of Health grant 5R01GM095476-04. Any opinions, ndings and con- clusions or recommendations expressed in this material are those of the authors and do not necessarily reect those of the sponsor. References 1. Agarwal, S., Yu, H.: Figsum: automatically generating structured text summaries for gures in biomedical literature. In: AMIA Annual Symposium Proceedings. vol. 2009, p. 6. American Medical Informatics Association (2009) 2. Bodenreider, O.: The Unied Medical Language System (UMLS): integrating biomedical terminology. Nucleic Acids Research 32(Database issue), D267D270 (Jan 2004) 3. Lavrenko, V., Croft, W.B.: Relevance based language models. In: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval. pp. 120127. SIGIR '01, ACM, New York, NY, USA (2001), http://doi.acm.org/10.1145/383952.383972 4. Lindberg, D.A., Humphreys, B.L., McCray, A.T.: The Unied Medical Language System. Methods of Information in Medicine 32(4), 281291 (Aug 1993) 5. Liu, F., Yu, H.: Learning to Rank Figures within a Biomedical Article. PLOS ONE 9(3) (MAR 13 2014) 6. Metzler, D., Croft, W.B.: A markov random eld model for term dependencies. In: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval. pp. 472479. SIGIR '05, ACM, New York, NY, USA (2005), http://dx.doi.org/10.1145/1076034.1076115 7. Metzler, D., Croft, W.B.: Linear feature-based models for information retrieval. Inf. Retr. 10(3), 257274 (Jun 2007), http://dx.doi.org/10.1007/s10791-006-9019-z 8. Yu, H., Liu, F., Ramesh, B.P.: Automatic gure ranking and user interfacing for intelligent gure search. PLoS One 5(10), e12983 (2010) 1308 Table 9. Retrieval parameters used by systems in dierent batches. Systems that only dier in the re-ranking share the same parameter settings. Dirichlet µ SDM Parameters λuni , λbi , λwbi RM Weight ω UMass-irSDM Batch 1 3000 0.58, 0.11, 0.11 0.19 Batch 2 2500 0.768, 0.004, 0.036 0.26 Batch 3 2500 0.768, 0.004, 0.036 0.26 Batch 4 2500 0.768, 0.004, 0.036 0.26 Batch 5 3000 0.72, 0.12, 0.16 0.005 Doc-Figdoc-UMLS Batch 1 3000 0.58, 0.11, 0.11 0.19 Batch 2 3000 0.58, 0.11, 0.11 0.19 Batch 3 2500 0.768, 0.004, 0.036 0.26 Batch 4 2500 0.768, 0.004, 0.036 0.26 Batch 5 2500 0.768, 0.004, 0.036 0.26 All-Figdoc-UMLS Batch 1 3000 0.58, 0.11, 0.11 0.19 Batch 2 3000 0.58, 0.11, 0.11 0.19 Batch 3 2500 0.768, 0.004, 0.036 0.26 Batch 4 2500 0.768, 0.004, 0.036 0.26 Batch 5 2500 0.768, 0.004, 0.036 0.26 All-Figdoc Batch 1 2500 0.768, 0.004, 0.036 0.26 Batch 2 2500 0.768, 0.004, 0.036 0.26 Batch 3 2500 0.768, 0.004, 0.036 0.26 Batch 4 2500 0.768, 0.004, 0.036 0.26 Batch 5 2500 0.768, 0.004, 0.036 0.26 All-Abstract-UMLS Batch 1 NA NA NA Batch 2 3000 0.56, -0.04, 0.04 0.36 Batch 3 3000 0.72, 0.12, 0.16 0.005 Batch 4 3000 0.56, -0.04, 0.04 0.36 Batch 5 3000 0.72, 0.12, 0.16 0.005 1309 Table 10. Examples of relevant snippets in PMC found within the top 2. 5319abb166e2b80600002f Which growth factors are known to be involved in the induction of EMT? in emt induction additionally non-smad signaling pathways activated by tgf-? and cross-talk with other signaling pathways including broblast growth factor fgf and tumor necrosis factor-? tnf-? signaling play important roles in emt promotion induction of emt in tumor stromal cells by (PMC 22111550, rank 1) 5319ac18b166e2b806000030 Is clathrin involved in E-cadherin endocytosis? plasma membranes we have found here that non-trans-interacting e-cadherin is constitutively endocytosed like integrin ligand-independent endocytosis that the formation of endocytosed vesicles of e-cadherin is clathrin dependent and that e-cadherin but not other cams at ajs and tjs including nectins claudins and occludin is selectively sorted into the endocytosed (PMC 15263019, rank 1) 5319abc9b166e2b80600002d Is Rac1 involved in cancer cell invasion? cells was clearly demonstrated by rna interference assay rac1 depletion signicantly suppressed the frequency of invasion in both quiescent and igf-i-stimulated mda-mb-231 cells this indicates the necessity of rac1 for igf-i-induced cell invasion in the cells overexpression of rac1 has been (PMC 21961005, rank 1) 5311bcc2e3eabad021000005 Describe a diet that reduces the chance of kidney stones. stone promoters and inhibitors reducing deposition and excretion of small particles of caox from the kidney maintaining the antioxidant environment and reducing the chance of them being retained in the urinary tract number of herbal extracts and their isolated constituents have also shown (PMC 23112535, rank 1) for age study on the relationship of an animal-rich diet with kidney stone formation has shown that as the xed acid content of the diet increases urinary calcium excretion also increases the inability to compensate for animal protein-induced calciuric response may be risk factor for the (PMC 21369385, rank 2) 530cf4fe960c95ad0c000003 Could Catecholaminergic Polymorphic Ventricular Tachycardia (CPVT) cause sudden cardiac death? case of catecholaminergic polymorphic ventricular tachycardia introduction in reid et al.1 discovered catecholaminergic polymorphic ventricular tachycardia cpvt cpvt is known to cause syncope or sudden cardiac death and the three distinguishing features of cpvt has subsequently been described (PMC 19568611, rank 1) 52fe58f82059c6d71c00007a Do archaeal genomes contain one or multiple origins of replication? genomes in the genus bacillus such positive correlation cannot be explained by the pure c?u/t mutation bias archaeal genomes multiple replication origins are typically assumed for archaeal genome replication multiple origins of replication implies multiple changes in polarity in nucleotide (PMC 22942672, rank 1) 52e204a998d0239505000012 Which is the denition of pyknons in DNA? processed the sequences of the human and mouse genomes using the previously outlined pyknon discovery methodology see methods section as well as ref and generated the corresponding pyknon sets by denition each pyknon is recurrent motif whose sequence has minimum length minimum number of intact (PMC 18450818, rank 1) 52d8494698d0239505000007 Which genes have been found mutated in Gray platelet syndrome patients? nbeal2 is mutated in gray platelet syndrome and is required for biogenesis of platelet alpha-granules platelets are organelle-rich cells that transport granule-bound compounds to tissues throughout the body platelet ?-granules the most abundant platelet organelles store large proteins that when released promote platelet adhesiveness haemostasis and wound (PMC 21765412, rank 1) 52ce531f03868f1b06000031 Are retroviruses used for gene therapy? frequently employed forms of gene delivery in somatic and germline gene therapies retroviruses in contrast to adenoviral and lentiviral vectors can transfect dividing cells because they can pass through the nuclear pores of mitotic cells this character of retroviruses make them proper candidates (PMC 23210086, rank 2) 1310