This paper presents the participation of Macquarie University and the Australian National University for Task B Phase B of the 2020 BioASQ Challenge (BioASQ8b). Our overall framework implements Query focused multi-document extractive summarisation by applying either a classi cation or a regression layer to the candidate sentence embeddings and to the comparison between the question and sentence embeddings. We experiment with variants using BERT and BioBERT, Siamese architectures, and reinforcement learning. We observe the best results when BERT is used to obtain the word embeddings, followed by an LSTM layer to obtain sentence embeddings. Variants using Siamese architectures or BioBERT did not improve the results.
Query focused multi-document summarisation aims to generate the answer to
a question by combining information from multiple documents [
The BioASQ Challenge3 organises shared tasks centered on biomedical texts. The focus of this paper is on the 2020 participation of Macquarie University and the Australian National University in Task B Phase B (BioASQ8b), where the aim is to nd the \ideal answer" to a question, given a collection of relevant PubMed abstracts.4 We approach this task as an instance of query focused multi-document extractive summarisation by scoring each candidate sentence and selecting the top-scoring ones to produce the nal summary. Macquarie University and the Australian National University submitted independent runs to BioASQ8b, but we both used the starting code of Macquarie University's BioASQ7b participation [10]. Novel contributions of this paper, compared with previous participation at BioASQ, include: 1. The incorporation of BERT and BioBERT in the general architecture of [10]. 2. The use of Siamese architectures in two main setups: 1) sharing weights in the architecture of [10], and 2) using Sentence-BERT [11]. 3. The use of Proximal Policy Optimisation (PPO) and BERT in a Reinforcement Learning approach.
This paper is structured as follows. Section 2 describes the general framework and baselines based on previous participation at BioASQ. Section 3 explains the incorporation of BERT and BioBERT. Section 4 introduces the incorporation of Siamese architectures. Section 5 describes the use of Reinforcement Learning. Section 6 presents and discusses the results of cross-validation evaluations using the BioASQ8b training data. Section 7 presents and discusses the runs submitted to BioASQ. Finally, Section 8 concludes this paper. 2
The overall architecture of our systems is based on that of Macquarie University's participation to BioASQ7b [10]. This architecture is described in Fig. 1. The baseline systems re-use the following options from [10] to generate the embeddings of the words and sentences.
{ The embedding generator to obtain word embeddings is a matrix of pre-trained embeddings generated by word2vec. We trained word2vec using PubMed documents provided by the organisers of BioASQ. The embeddings had a vector size of 100. { The embedding reductor to obtain sentence embeddings is a pair of forward and backward LSTM chains. The weights of the embedding reductor for the candidate sentence and the question were not shared. { The activation function of the nal layer is a linear function for the regression setup and a sigmoid function for the classi cation setup. 3 http://www.bioasq.org 4 http://www.ncbi.nlm.nih.gov/pubmed embedding generator embedding reductor (siamese) embedding reductor relu linear (NNR) sigmoid (NNC) sentence position word embeddings { The similarity between the embeddings of the candidate sentence and the question is the element-wise product.
As in [10], we tried a classi cation setup and a regression setup. In the regression setup (\NNR" in Fig. 1 Table 3), the training data is labelled with the ROUGE-SU4 F1 value of the candidate sentence and the objective function to optimise is the Mean Squared Error. In the classi cation setup (\NNC" in Fig. 1 Table 3), the 5 candidate sentences with the highest ROUGE-SU4 F1 are labelled as 1, and the rest are labelled as 0. The objective function to optimise in the classi cation setup is binary cross-entropy.
In both the classi cation and the regression setup, the summary is produced by scoring each candidate sentence and extracting the top n sentences to generate the summary, where n depended on the question type and was the same as reported in [10] (Table 1).
BERT [
In a rst experiment (\BERT untrained" in Table 3), we replaced the embedding generator with BERT using the pre-trained model provided by Huggingface.5 The resulting word embeddings are now a ected by context. The BERT weights were not updated during the training stage. Following the recommendation of the Huggingface library, the embedding reductor of each candidate sentence and the question are the average of the word embeddings.
In a subsequent experiment (\BERT trained" in Table 3), the embedding generator and reductors are as in BERT untrained, but we allowed the BERT weights to be ne-tuned during the training process.
We also tried a variant (\BERT LSTM" in Table 3) that uses BERT in the embedding generator as in BERT untrained, but the embedding reductor is a bidirectional LSTM chain as in the NNC baseline. This variant is therefore comparable with the NNC baseline, and the only di erence being the use of BERT for the embedding generator.
In a nal series of experiments (\BioBERT untrained" and \BioBERT LSTM" in Table 3), the embedding generator and reductors are as in BERT untrained and BERT LSTM, but the pre-trained model was as provided by the developers of BioBERT6 [6], who used biomedical documents to pre-train BERT.
In all of the experiments in this section, the nal layer was a classi cation layer with a sigmoid activation and the objective function to optimise was binary cross-entropy. 4
Siamese networks have been used in applications that include a comparison between documents, for example to determine semantic similarity [11]. The general idea is to use the same processing module for each of the two documents by sharing the weights of the encoding component that generates the embeddings of the documents. We have used this idea in two main kinds of experiments described below. 4.1
A straightforward implementation of Siamese Networks (\Siamese LSTM" in Table 3) using the overall architecture of Fig. 1 shares the weights of the embedding reductors of the candidate sentence and the question. This ensures that the sentence embeddings are generated using exactly the same process.
4.2
The second implementation of Siamese Networks uses Sentence-BERT (SBERT) [11]
to determine whether a candidate sentence is similar to the question. In
particular, the system uses BERT [
difference regr SBERT cos j j softmax
R clas
Depending on the settings, the system uses regression, classi cation or both (multi-task), for prediction.
{ In the regression setup, the cosine similarity between candidate sentence and question embeddings is computed, and the objective function to optimise is the Mean Squared Error (MSE). { In the classi cation setup, the candidate sentence and question embeddings are concatenated with the element-wise absolute di erence between the sentence and question, and a softmax layer is added. The objective function to optimise is binary cross-entropy. { In the multi-task setup, the classi cation and regression setups are jointly optimised during training. At prediction time, we use either the classi cation head or the regression head.
In both the regression and the classi cation setups, the training data was labelled with the classi cation labels mentioned in Section 2.
We also experimented with a BERT and a BioBERT variation7 of a Sentence Transformer [11] to ensure that the sentence embeddings re ect a biomedical word-space.
Table 2 shows all the combinations we tried.
System Name Model SBERT R SBERT C SBERT M R SBERT M C BERT BERT BERT BERT SBioBERT R BioBERT SBioBERT C BioBERT SBioBERT M R BioBERT SBioBERT M C BioBERT Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y Y
Reinforcement learning allows the training process to optimise the target evaluation metric (ROUGE-SU4 F1) directly. Whereas [10] used the REINFORCE algorithm [13], in our participation to BioASQ8b we used the Proximal Policy Optimisation (PPO) approach. We choose PPO for our summarisation task because past research shows that it penalises changes to the policy [12], and because we observe a more consistent learning curve using this approach compared to REINFORCE [5]. 5.1
As in past submissions, our reinforcement learning system classi es sentences to
be either (0) not included in the summary or (1) included in the summary based
on a policy, using the ROUGE-SU4 F1 score directly as the reward. We use
the stable baselines library8 [
We perform some hyperparameter tuning, while leaving the PPO code unmodi ed as much as possible. We use a horizon (n steps) of 1000 with 4 minibatches each, and run for a total of 500,000 timesteps. When training our model, we choose any action from the probability distribution of the policy function, but when testing we choose the action out of 100 samples.
The neural network architecture for PPO is shown in Figure 3. As in [10], the inputs of the neural network consist of: 1. Candidate sentence 2. Question 3. Summary generated so far 4. Sentences after the candidate sentence 5. Entire document 8 https://stable-baselines.readthedocs.io/ sentence question summary next sentences document summary length 6. Length of summary generated so far
For each of the rst ve inputs, we take the mean of the word2vec word embeddings (each of size 100) to generate the sentence embeddings. The sentence embeddings and the length of the summary are concatenated to form a single layer of size 501. We then feed the combined layer into a simple Multi-Layer Perceptron with two hidden layers each of size 200. The outputs of this neural network become the value function and stochastic policy function for our PPO approach, each of size 2 with linear activations, representing the predicted future rewards and action probabilities respectively for the two actions (classify 0 or 1). word embeddings We also apply pre-trained BERT embeddings to our reinforcement learning approach for batches 4 and 5 of BioASQ8b. Our network architecture is the same as in Figure 3, but we change the embedding generator to generate BERT embeddings instead of word2vec embeddings. We use the PyTorch BERT embedding generator provided by Huggingface5 and the TensorFlow Multi-Layer Perceptron policy provided by stable baselines8.We observe a minor improvement in ROUGE-SU4 F1 score as shown in Section 7 (PPO BERT).
All experiments except those described in Sections 5 were evaluated using 10-fold cross-validation using the training data provided by the organisers of BioASQ8b. These results are shown in Table 3. As described in Section 6.2, cross-validation evaluation was not practical for the experiments with reinforcement learning.
The baseline runs \ rstn", \NNR" and \NNC" con rm [10]'s observation that the classi cation setup produces better results than the regression setup. The untrained BERT system did not better the classi cation system, and the trained BERT system produced worse results.
We observed no changes in the evaluation results of the BERT trained system as we changed the number of epochs from 1 to 20 epochs. A detailed look at the changes in the loss during training revealed a very small improvement of the loss as we increase the number of epochs, but not enough to re ect a di erence in the nal ROUGE-SU4 F1 metric. This suggests that a more elaborate netuning regime with gradual unfreezing as described by [4] might lead to improved results.
Given the poor results of the trained BERT system, we kept BERT frozen when we tested the variants using LSTM and using BioBERT.
BERT and BioBERT followed by an LSTM-based sentence reductor did improve results over the version with a mean of embeddings. Unfortunately, the experiments using the LSTM-based reductor were made after the deadlines for submission of results to BioASQ8b.
It was surprising to observe that the BioBERT variants performed worse than the BERT variants. This is not in line with the improvement in the performance of BioBERT for the \exact answers" section of BioASQ7b reported in literature [14].
The Siamese LSTM variant did not improve over the classi cation system. The reason for this might be, as mentioned in Section 7, that questions and candidate sentences are di erent in nature. We also conducted cross-validation experiments with the SBERT variants but they are not included in Table 3 as the results are very di erent from those of the runs submitted to BioASQ (Section 6) and we suspect that there might have been an error when running these evaluations.
Table 4 shows and explains the hyperparameters of all systems of Table 4.
Batch size Dropout Epochs Sentence length 6.2
The evaluation setup for reinforcement learning did not use cross-validation due to the long time it took to train the system (several days). We therefore used a partition of the training data for the train and test sets.
Method NNR NNC BERT untrained BERT trained BERT LSTM BioBERT untrained BioBERT LSTM Siamese LSTM 1024 1024 32 8 1024 1024 1024 1024 0.3 0.3
The BioASQ7b data set consists of 2,747 questions which we divide into a training set (2,289 questions) and a testing set (458 questions) partitioned 5:1 using a random seed to shu e the data. The PPO word2vec and REINFORCE word2vec systems were run 3 times each for 500,000 timesteps on the BioASQ7b data set, and the maximum ROUGE-SU4 F1 score reached in each learning curve was averaged across the 3 runs.
The BioASQ8b data set consists of 3,243 questions which we divide into a training set (2,702 questions) and a testing set (541 questions) partitioned 5:1 using a random seed to shu e the data. The PPO word2vec, PPO BERT, and REINFORCE word2vec systems were each run once only for 500,000 timesteps on the BioASQ8b data set, and the maximum ROUGE-SU4 F1 score for each run is shown in Table 5. The models which produced the maximum ROUGE-SU4 F1 score for each system were saved and re-used to generate our PPO submissions to the BioASQ competition in 2020, except for the REINFORCE system which was not included in this year's submission.
Whereas our evaluation results did not show any clear di erence between REINFORCE and PPO on word2vec embeddings, the version using PPO and BERT features showed an improvement.
The reinforcement learning experiments of Table 5, however, do not outperform the results of the experiments of Table 3. 7
All runs in Table 6 have been described except for First nb in batch 2. This is the same as First n in all other batches but the data has been pre-processed di erently.
The classi er system (NNC) produces the best results in most runs, and this is consistent with our cross-validation results (Table 3). However, in contrast with Table 3, in most runs the second best is the regression system (NNR) instead of the untrained BERT system. All systems were re-trained for the BioASQ runs, keeping the same hyperparameters, so that we could use the entire training data, and it is possible that bad luck played a part here, and the BERT system was not trained to its best.
The reinforcement learning experiments show the lowest evaluation results. We should note, however, that the RL runs submitted to BioASQ7b also had lower results than the other runs, but the human evaluation results ranked them higher than our other runs. Also, the reinforcement learning submissions were not re-trained using the entire training data, which may be worth exploring because PPO had a more consistent learning curve for us than REINFORCE. We are waiting for the human evaluation results of BioASQ8b with anticipation.
Table 7 shows the results of the Australian National University's submissions to BioASQ8b.
We observe that the SBERT runs perform worse or on par with the BERT untrained model of Table 6. This might be because questions and answers may not appear closely in an embedding space, and therefore an identical processing of each of them might not be advantageous. For example, a question includes question words whereas the candidate sentences are not normally questions. We also observe that the BioBERT models seem to produce lower results when compared with the BERT untrained model of Table 6. In general, the multitask learning approach performed better than just classi cation or regression. 8
This paper presents our approaches to query focused multi-document extractive summarisation for BioASQ8b. Our experiments include the use of BERT and BioBERT, Siamese architectures and SBERT, and Reinforcement Learning with PPO.
We observed that an approach that uses BERT to obtain the word embeddings, followed by LSTM to map these word embeddings to sentence embeddings, had the most promising results. The variant with BioBERT did not present an improvement, and this con icts with the overall improvement of the use of BioBERT for question answering on biomedical texts. The approaches with Siamese architectures did not present an improvement over the base versions, presumably because questions and candidate sentences have di erent sentence styles.
As further research we plan to explore ne-tuning techniques for the BERT and BioBERT variants.
Research by Vincent Nguyen is supported by the Australian Research Training Program and the CSIRO Postgraduate Scholarship. 4. Howard, J., Ruder, S.: Universal language model ne-tuning for text classi cation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 328{339. Melbourne, Australia (Jul 2018). https://doi.org/10.18653/v1/P18-1031, https://www.aclweb. org/anthology/P18-1031 5. Jones, C.R.: Reinforcement Learning For Query-based Multi-document Extractive
Summarisation. Master's thesis, Macquarie University (Jan 2020) 6. Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C.H., Kang, J.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234{1240 (09 2019), https://academic.oup.com/ bioinformatics/article/36/4/1234/5566506 7. Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., Zettlemoyer, L.: BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 7871{7880. Online (Jul 2020), https://www.aclweb.org/anthology/2020. acl-main.703 8. Liu, Y., Lapata, M.: Text summarization with pretrained encoders. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. 3730{3740. Hong Kong, China (Nov 2019). https://doi.org/10.18653/v1/D19-1387, https://www.aclweb.org/ anthology/D19-1387 9. Mekala, D., Shang, J.: Contextualized weak supervision for text classi cation. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. pp. 323{333. Online (Jul 2020), https://www.aclweb.org/anthology/ 2020.acl-main.30 10. Molla, D., Jones, C.: Classi cation betters regression in query-based multidocument summarisation techniques for question answering. In: Cellier, P., Driessens, K. (eds.) Machine Learning and Knowledge Discovery in Databases. pp. 624{635. Springer International Publishing, Cham (2020) 11. Reimers, N., Gurevych, I.: Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). pp. 3980{3990. Stroudsburg, PA, USA (2019). https://doi.org/10.18653/v1/D19-1410, https://www.aclweb. org/anthology/D19-1410 12. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017) 13. Williams, R.J.: Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8(3-4), 229{256 (1992) 14. Yoon, W., Lee, J., Kim, D., Jeong, M., Kang, J.: Pre-trained language model for biomedical question answering. In: Cellier, P., Driessens, K. (eds.) Machine Learning and Knowledge Discovery in Databases. pp. 727{740. Springer International Publishing, Cham (2020)