This paper presents query-sentence matching based and UMLS query-graph based summarization techniques for query-speci c biomedical text summarization. The query-speci c graphs, constructed using UMLS entities and relations, are used for matching the sentences. The core idea is to nd candidate biomedical entities for query expansion which are semantically connected. The graph represents these connections and it was automatically constructed using UMLS knowledge source and biomedical text. The results of the proposed techniques experimented on previous BioASQ dataset are better as compared to the results of baseline techniques. The same techniques are applied on task 8B dataset for ideal answer generation and submitted to BioASQ8. These submitted results gave the highest scores among all participants' submissions for automatic evaluation scores(ROUGE-2 Recall and ROUGE-SU4 Recall).
UMLS
Query-focused
Biomedical text and information on the web are growing exponentially nowadays.
Text summarization attempts to provide the users with a summarized version of
the text with maximum information content in a compact, quick and intelligible
way. In recent years, substantial research has been conducted to develop and
evaluate various summarization techniques in the biomedical domain. Recent
research has focused on a hybrid technique comprising statistical, language
processing and machine learning techniques [
Automatic text summarization of biomedical text is a promising method for helping clinicians and researchers to e ciently obtain and understand any topic by producing a summary from one or multiple documents. The goal of text summarization is to present a subset of the source text, which expresses the most important points with minimal redundancy. Thus, text summarization may become an important tool to assist clinicians and researchers with their information and knowledge management tasks. Sometimes, there may exist a query for which the user seeks information and sometimes it may not. In the case of query-focused summarization, the generated summary should have the answer to the query in it. It is a very usual scenario that users want exact answers along with some related details in case of their medical related queries. Therefore, we are focusing here on query-focused biomedical multi-document summarization which will be helpful to clinicians and all other users who are seeking elaborated answers to their medical related queries.
BioASQ1 organizes challenges which include biomedical semantic indexing and question answering. The question answering task uses benchmark datasets containing development and test questions, in English, along with gold standard (reference) answers constructed by a team of biomedical experts. The participants have to respond with various types of answers. Speci cally, task B has questions with their related documents and snippets for which exact answers and ideal answers need to be generated. Here we focus on generating ideal answers for the questions. The ideal answers are paragraph sized summaries with multiple sentences. We are focusing on generating ideal answers using extractive summarization on available snippets.
The remainder of this paper is presented as follows: Section 2 shows the related works. Section 3 describes the baseline methods and the proposed methods for query-focused biomedical text summarization. Sections 4 presents the experiments and results with analysis. Finally, section 6 concludes it. 2
A lot of research has been carried out in the eld of biomedical text
summarization. A recent survey on the research in text summarization in the biomedical
domain highlights that natural language processing and hybrid techniques were
prominently used for summarization of multiple documents [
The graph-based summarization using named-entities has been presented
as EntityRank algorithm which considers information about named entities in
the process of multi-document graph-based summarization [
Query based biomedical text summarization techniques that rely on external
ontology knowledge resource UMLS are proposed in the literature [
Text summarization approaches often rely on the similarity measure to model
the text documents. [
Here we propose an approach for query-speci c biomedical text summarization
which uses ontology knowledge source UMLS [
3.1
This section describes baseline summarization methods, query sentence matching based method, modi ed query sentence matching based method using UMLS query graph and modi ed lexrank using UMLS query-graph.
Two basic approaches of summarization TextRank [
P tfw;s1 tfw;s2(idfw)2 w2s1;s2 sim(s1; s2) = r P (tfw;s1 idfw)2r P (tfw;s2 idfw)2 w2s1 w2s2 (1) where tfw;si is the number of occurrences of the word w in the sentences si.
In the graph, edges were formed between the sentences having similarity greater than the threshold. In both algorithms, the sentences are ranked by applying PageRank to the resulting graph. A summary is formed by combining the top ranking sentences, using a length cuto to limit the size of the summary. 3.2
The UMLS query-graph based lexrank is a modi ed version of lexrank which uses query-speci c graphs generated using UMLS to get the importance of words, matches sentences using weighted cosine similarity measure, generates a graph of sentences and then applies pagerank on the graph.
Query-speci c graph has been generated with the use of UMLS entities and
relations as described in [
Two nodes in the graph can have an edge between them if and only if those two entities have some relation in UMLS. There are various types of relations present in UMLS and all types of relations are used. There can be some isolated nodes in the graph when any query concept is not related to any other query concept or it does not have any common related concept with another query concept.
The graph is further re ned by assigning weights to the edges and removing some of the edges in the graph. The edge weights are calculated based on the co-occurrence value of entities in the text to be summarized. For any edge between two entities, the co-occurrence value of two entities is used as the weight for that edge. The edges whose edge weights are less than some threshold are removed from the graph. Less edge weight for an edge between two entities means those two entities rarely occur together and hence share very less or no context.
In the re ned graph, the nodes are weighted using PageRank [
The main di erence with lexrank method is UMLS query-graph weighted cosine similarity. The later processing is same as it is in lexrank. The UMLS query graph based weighted cosine similarity is: where,
Ww = importance of concept w from query-graph, if w is in query-graph = 0, otherwise tfw;q and tfw;s are the number of occurrences of the word w in query q and sentence s, respectively. idfw is the inverse of the number of sentences in which word w is present.
The formula incorporates the weights of the extended query terms from query graph in the tf-idf vectors of every sentence containing those terms. The Query Sentence Matching (QSM) based summarization method compares all the sentences with the query and takes top similar sentences to query as summary. The queries and all the sentences in snippets are represented by vectors of tf-idf values of words in the sentences. The similarity measure used to match query vector and sentence vector is cosine similarity as given by equation 1. The only di erence here is that the similarity is calculated between query and a sentence instead of similarity between two sentences.
The UMLS querygraph QSM summarization method is a modi ed version of QSM
which uses query-speci c graphs generated using UMLS to get the importance
of words. For each query, it generates a query-speci c graph as described in
[
P (tfw;qidfw+Ww;q)(tfw;sidfw) w2q;s sim(q; s) = r P (tfw;qidfw+Ww;q)2r P (tfw;sidfw)2
w2q w2s where,
Ww;q = importance of concept w from query-graph of q, if w is in query-graph = 0, otherwise tfw;q and tfw;s are the number of occurrences of the word w in query q and sentence s, respectively. idfw is the inverse of the number of sentences in which word w is present.
Here, the weights are only considered for query vector. They are not incorporated in sentence vectors unlike UMLS graph based lexrank. The intuition for updating only query vector was to see it as an query expansion procedure for query-focused text summarization. 4
This section describes the experiments performed along with their results. For our
experiments, we have used the dataset of BioASQ task 5B phase B and BioASQ
task 8B phase B. BioASQ task 5B phase B dataset is used as a benchmark dataset
which contains various questions in English, along with gold standard (reference)
answers constructed by a team of biomedical experts. The test dataset has ve
di erent batches, each containing 100 questions. For each question, the relevant
snippets are given and the ideal answer for that question needs to be generated.
The ideal answers are paragraph sized summaries so it's a case of multi-document
summarization on relevant snippets. The evaluation is done using The ROUGE
[
The results on all 5 batches of BIOASQ 5B phase B dataset are presented here. Table 1 and table 2 show a comparison of summarization methods (described in previous section) in terms of ROUGE-2 Recall and ROUGE-2 F-measure, respectively. Table 3 and table 4 shows a comparison of summarization methods in terms of ROUGE-SU4 Recall and ROUGE-SU4 F-measure, respectively. The results show that UMLS querygraph QSM gives an improvement over QSM. The method lexrank UMLS querygraph gives an improvement over lexrank for batch 1,3 and 5 of the dataset. For the other two batches, the results are comparable. For batch 5, the ROUGE-2 Recall and ROUGE-2 F-measure results of lexrank UMLS querygraph are statistically signi cantly better than lexrank.
The graphs in the rst row of g. 1 shows the query type wise change in the results of lexrank UMLS querygraph as compared to lexrank for every batch of the data while the second row shows the batch wise distribution of the queries based on their types. From the graphs, we can say that the 'yesno' type of questions are getting improved in all batches (considering batch 2 where it is showing zero change: no improvement and no deterioration). The graph of batch 5 indicates that the major part of e ectiveness of the method lexrank UMLS querygraph comes from the improvements in 'factoid' and 'yesno' type of queries with a small contribution from 'summary' types of the queries. For batch 2 and 4 where lexrank UMLS querygraph failed, decrements in 'factoid', 'list' and 'summary' type of queries must be the reason. 4.3
Among all participants' submitted runs, these ve submitted runs appeared to be top ve(DAIICT QSM being the highest) for ROUGE-2 Recall and ROUGESU4 Recall. For ROUGE-2 F-measure, DAIICT QSM is second highest considering all participants' runs and it is the third highest in case of ROUGE-SU4 F-measure. 5
This paper presents query sentence matching based summarization techniques and UMLS query graph based summarization techniques which were submitted to BioASQ8 challenge for ideal answer generation in task B on biomedical semantic question answering. These techniques incorporate weights of the candidate biomedical entities from queries and their semantically related entities identi ed by UMLS and do the matching using tf-idf vectors. The results of the proposed techniques on BioASQ 5B phase B dataset are compared with baselines textrank and lexrank. The analysis shows that the UMLS query graph based method gives comparable results with the baselines and helps to improve 'yesno' type of questions. The results of these techniques on BioASQ task 8B phase B batch 5 dataset were the highest among all participants where simple QSM approach outperformed UMLS graph based QSM as well as UMLS graph based lexrank.