The recognition of the functional structure can help to understand the logical structure and content structure of the text, understand the text from a deep level, and further realize the academic big data analysis and data mining. In this study, the academic full text is taken as the research object, and the papers published in JASIST journals from 2010 to 2020 are selected to conduct a comparison experiment on the four models of Prompt, SciBERT, LSTM and TextCNN. The results showed that Prompt model had the best recognition effect.
Automatic recognition of the functional structure of academic texts is an important issue in the field of natural language processing ( Lu, et al., 2018 ). On the one hand, it can make the logical structure of a paper clearer and present in a finegrained way, which can enable researchers to retrieve the required literature information faster and save time. On the other hand, it helps to standardize the structure of the paper. At present, machine learning and deep learning have become the mainstream methods of paragraph structure recognition.
Traditional machine algorithms mainly include support vector machine, naive Bayes, Logistic regression and K- nearest neighbour algorithm. TUAROB et al. (2015) compared two models, naive Bayes and support vector machine, to divide chapter boundaries of academic literature. Since Hinton et al.(2006) put forward the concept of deep learning in 2006, in addition to these models based on traditional machine learning, the research on the use of deep models has gradually deepened. Tkaczyk et al. (2015) proposed a system based on modular open source flow to extract metadata in the paper. However, the proposal of pre-training language model makes the representation effect of word vector to a higher level. Ren et al. (2017) fully studied the structural features of academic texts, and combined with convolutional neural network, proposed an automatic functional structure detector to identify the structure of academic texts. The BERT model is excellent, but fine-tuning requires a lot of data and computational power, and not enough data is available for all scenarios. The prompt based downstream tasks have recently become a boon to small sample learning. Prompt allows the model input to be modified to bring downstream tasks closer to the pre-trained model.
Therefore, Prompt, SciBERT, LSTM and TextCNN models were selected in this research for comparative experiments. The optimal recognition model is sought and the influence of different models on structure function recognition is discussed.
This research obtained all the full texts of academic papers published in Journal of the Association for Information Science and Technology (JASIST ) from 2010 to 2020 by using self-made Python program.
As for data annotation, in this paper, the collected 2232 articles are divided into text according to paragraph as the basic unit, and the structure of these paragraphs is marked. Combined with previous research on the functional structure of academic texts, this paper divides the structure and function of academic texts into five parts: "introduction", "relevant research", "method", "experiment" and "conclusion", which are represented by "I", "R", "M", "E" and "C" respectively. The specific labeling process is as follows: BERT model is first invoked to complete automatic functional structure labeling of academic full-text data from 2010 to 2020. In order to ensure the accuracy of labeling, manual verification is required. After manual review and collation, preliminary text data is obtained. The number of marked paragraphs of each structure is shown in Table1 below . Table 1 Basic Information of the Corpus
Prompt reconstructs the template for different tasks, inputs human-made rules into the pre-training model, and makes the model better understand human instructions, bridging the gap between the training process and downstream tasks. SciBERT is a BERT pre-trained using a total of 1.14 million scientific papers in biomedical (82%) and computer science (12%) directions and may be more suitable for natural language processing tasks in the direction of scientific papers. LSTM is a variant of RNN. RNN can only have Short-Term Memory due to the disappearance of gradient. Compared with RNN network, LSTM network combines short-term memory with long-term memory by adding additional state c and using gate control. And to some extent, it solves the problem of disappearing gradient. TextCNN, as the name implies, is CNN for text tasks. Each word is mapped to a word vector by embedding, and then input to sofxmax layer through convolution layer and max-pooling layer to realize text classification.
The main parameters of Prompt are as follows:
epochs is 10, learning_rate is 1e-5, max_len is
512.The main parameters of SciBERT are as
follows: max_len is 128, epochs is 3, learning_rate is
2e-5, num_attention_heads is 12, hidden_size is
300. LSTM parameters are as follows: max_len is
512, epochs is10, and learning_rate is 0.005. The
main parameters of TextCNN are as
follows :epochs is 5, filter_sizes is (
Table 2. Results of 10-Fold Cross-Validation
From the perspective of various structural
functions, the effect of the introduction is the best, and
the average of the three indicators can reach 95%,
followed by the conclusion and method, and the
effect of related research is the worst. The reasons
are as follows: (
In the experiment of functional structure recognition of the full text, the overall performance of the functional structure Prompt on the test set is the best, because the model input the prompt information, can be more fully mining the semantic knowledge in the pre-training model. Secondly, from the view of the recognition effect of each structure, Prompt has the best performance in the introduction, related studies, methods, experiments, conclusions, showing the strong learning ability of the model, as well as a wide range of application prospects.