Introduction

With the progress of science and technology, there are more and more fields and scientific articles. Information extraction and knowledge mining in the specific field enable scholars to quickly grasp the overall outline of information, and track the development of fine-grained knowledge. There are many mature models to extract information from texts, such as BiLSTM-CNN [1], CNN-BiLSTM-CRF [2], LM-LSTM-CRF [3], which have achieved high scores in various tasks of natural language processing. In fact, these supervised learning models inevitably consume large amounts of high-quality annotated corpus in order to fully learn the characteristics of natural language representation. In most case, however, the annotated corpus in one specific field is constructed manually by several experts, which is timeconsuming and laborious. Therefore, it is hard to directly use a well-trained model to other domains.

How to extract information without massive annotated corpus is a big challenge. Active Learning (AL) [4] has been proved to be an effective way to solve the problem of corpus scarcity when dealing with the classification tasks [5,6]. However, it has not been validated on the sequence labelling task, which is more difficult to find the optimal result because its complexity increases exponentially [7]. In this paper, we introduce multiple active learning strategies into information extraction for the first time, so as to explore a cheap and efficient solution for recognizing the fined-grain entities in multiple domains.

Relation predication is another basic technology for knowledge organization. Translation models see relation as a process of translating the head entity to the tail entity, which have been widely used to predict relations. There are some classic translation models proposed from different perspectives: TransE [8] is the first translation embedding model with fewer parameters. TransH [9] is presented to solve the problem of complex relation representation. TransR [10] distinguishes the semantic embedding for different types of relations, which wined a better F-score. TransD [11] simplifies the projection process of TransR and improves the computing efficiency. This paper aims to construct an intelligent platform for information extraction and knowledge mining, which can be used in multiple domains without much human intervention. The main contributions are as follows: 1). with the limited annotated corpus, an effective method combining neural network with active learning recognizes scientific entities in multiple domains; 2). By optimizing the loss function, an improved translation model represents the semantic vectors more accurately and reaches the convergence state faster with a small loss score compared with the original model. combining neural network with active learning extracts "problem" and "method" entities, 2) the improved translation model predicts relations between "problem" and "method" entities. At the same time, the platform integrates two excellent tools (spaCy.NER1 and RAKE2 ) to recognize the named entities and keywords. Finally, this platform provides a variety of knowledge services for researchers, including problem discovery, method recognition, relation representation and hot spot detection. Besides, the analyzers can perform richer downstream tasks based on our platform, such as discipling analysis, trend explosion, new technology detection, and so on.

Scientific Entity Recognition

Scientific entity recognition contributes to extract phrases from scientific articles. These phrases consist of several words which describe the focus of article or the method proposed by author. In order to reduce the dependence on annotated corpus, this paper provides a semi-supervised learning model combining neural network with active learning.

The framework of the information extraction model is shown in Figure 2. Firstly, the learning engine trains the parameters of neural network by using a small number of annotated samples (dozens of abstracts with semantic labels). Then, the trained neural network predicts the labels of unannotated samples and inputs the predicted scores to the selecting engine. Secondly, according to the active learning strategies, the selecting engine decides which samples are valuable and should be annotated manually. Only the top 10% most valuable samples are labelled by experts. Thirdly, the manually annotated samples are added into the training set to re-train the neural network, in order to improve the performance of label prediction. The whole process runs repeatedly until the performance of model has no significant optimization. Finally, the trained model predicts the "problems" and "methods" for all the unlabeled articles. More details about parameter setting will be discussed in Section 3.1.

Figure 2: Information extraction model combining neural network with active learning

Here we choose CNN-BiLSTM-CRF [12] as the learning engine. CNN focuses on the morphology features that are the prefix and suffix of word. BiLSTM learns the dependency relationship between words with a long distance by using two groups of long-short term memory networks in opposite directions. CRF decides the most optimal labeling sequence with a rational linguistic logic.

In addition, we propose a hybrid approach for the selecting engine. Firstly, the value score of each unlabeled sample is respectively computed by four different types of active learning strategies, and the sum of them is set as the final value score. Secondly, the value scores are listed in descending order, only the top 10% most valuable samples are selected to be annotated manually in each iteration. This paper picked out three classical strategies from the uncertain sampling methods: margin [13], N-best sequence entropy [14] and maximum normalized log-probability [15]. Additionally, we propose a novel strategy, namely label weighted probability, which enhances on the importance of the number of labels. The more labels of problems or methods there are in a sentence, the more valuable the sentence is.

Entity Relation Prediction

Relation prediction decides whether a "problem" and a "method" is related or not. That means if a "problem" is related to a "method", the method can be used to solve this problem.

Translation model sees the relation in the triple (head entity, relation, tail entity) as a translational between two entities. There is a series of translation models. TransE [8] has few parameters and is low in complexity, but cannot distinguish two tail entities with the same relation. TransH [9] uses different vectors to represent one entity with various relations, which solves the problem of complex relation representation (1-N, N-1, N-N). TransR [10] supposes that different relations are in different semantic spaces. Thus, this model projects entities into their relation spaces at first, then builds the translation process. However, it greatly increases the time cost because of too many parameters. TransD [11] creates the projection matrix respectively for head entity and tail entity. It not only combines the effects of both entities and relations on projection, but also improves the computing efficiency.

After comparing the performance of various translation models, we choose TransH to predict relations, which keeps balance between accuracy and efficiency. To solve the problems of oneto-many, many-to-one, many-to-many relations, TransH generates the relation-specific translation vector 𝑑 𝑟 in the relation-specific hyperplane 𝑤 𝑟 rather than in the same space of entity embeddings.

Figure 3: TransH projection [9]

As shown in Figure 3, the relation 𝑟 in its hyperplane 𝑤 𝑟 has a translation vector 𝑑 𝑟 , the head embedding ℎ and the tail embedding 𝑡 in 𝑤 𝑟 have their projection vectors ℎ ⊥ and 𝑡 ⊥ . The defined score function is:

||ℎ ⊥ + 𝑑 𝑟 − 𝑡 ⊥ || 2 2 .

However, the original TransH model does not match our goal exactly. We achieved three improvements.

1) TransH constructs the negative samples by replacing the head or tail entity with others in the positive samples.

However, the replaced one may also be correct because of synonyms, which introduced many false negative labels into training. Considering that there are only two types of relationships, we simply construct the negative samples by modifying the correct relationship into its antonym. By this change, it is more convenient to construct a balanced annotated corpus. Moreover, the score function 𝑓 𝑟 (ℎ, 𝑡) is redefined as Equation (1), which aims to move the attention from entity to relation.

𝑓 𝑟 (ℎ, 𝑡) = ||𝑎𝑏𝑠(ℎ ⊥ − 𝑡 ⊥ ) − 𝑑 𝑟 || 2 2(1)

2) Comparing with the original model that initializes the entities with the random vectors, we use the word2vec model to generate the semantic representation of all head and tail entities.

3) To improve the ability of feature learning for the unknown entities, we add one hidden layer of linear transformation respectively for the head entities and tail entities.

Named Entity Recognition and Keyword Extraction

We use an enterprise open source toolkit spaCy.NER to recognize the named entities. spaCy.NER implements a very fast and efficient system based on the statistical machine learning algorithms, which can recognize 18 entity types, such as Person, Organization, Location, Geopolitics entity.

Furthermore, keyword extraction is achieved by the open source toolkit RAKE (Rapid Automatic Keyword Extraction). RAKE is an automatic keyword extraction technique. Based on the statistical method, RAKE outperformed TextRank and other supervised learning models, which obtained a high F value [16] and is more efficient.

Platform Evaluation and Display

We evaluate the performance of information extraction of IEKM-MD in the field of Artificial Intelligence (AI). There are two datasets be used.

1) The top 100 AI conferences were picked out by the domain experts, and their abstracts were acquired from NSTL database3 , total in 9753 sentences. Next, we built the truth datasets. Each sentence is annotated synchronously by two students in the corresponding subjects (task, method or other). The annotation results are checked by one expert. The annotation format is shown as Figure 4. The AI annotated corpus contains 26,0000 tokens. In addition, we show the effect of knowledge mining in three different kinds of domains. We choose three popular keywords (Neural Networks, Nano Structure and Genetic Engineering) that respectively respect the subjects of Computer Science, Material and Medicine to acquire abstracts from NSTL database. 200

h t w r h ⊥ t ⊥ d r

abstracts of each subject are randomly selected from SCI journals and are used to verify the practical application effect of IEKM-MD.

Scientific Entity Recognition

We set the baselines only using the CNN-BiLSTM-CRF (CBC) model trained on all annotated samples. For each dataset (AI or FTD), the best performance is as the baseline, so as to detect whether active learning helps reduce the scale of annotated corpus for supervised learning models. The scale of training sets and the best F1 scores of CBC model are shown in Table 1. In the model of IEKM-MD, initially only 0.01% annotated samples are used to carry out the cold starting process, then the highest valuable samples (10%) are added into the training sets in each iteration. Only if the F1 score of IEKM-MD reaches the baseline, can the learning process be stopped. The label scales and F1 scores of AI and FTD datasets in each iteration are show in Table 2.

Table 2: Learning effect of IEKM-MD in each iteration

Step 1 and 2 reveal that after combing supervised learning model with active learning strategies, the annotated samples can be cut down 60%-70%.

After IEKM-MD achieves the best performance as that CBC model did, the model extracts problems and methods from Neural Networks, Nano Structure and Genetic Engineering datasets. We manually checked the top 30 problems and methods and evaluated their accuracies as shown in Table 3.

The results reflect that Neural Networks achieved the best performance with 0.93 accuracy of problem extraction and 0.89 accuracy of method extraction. The average accuracy of three fields reveals that problem extraction has a better score than method extraction. The first reason is that the total mentions of problem are smaller than methods, and they are usually described in the noun phrases, which contribute to an easier pattern to be caught by model. The second reason is that one article may contain multiple methods, which are modified by multiple attributives or adverbials, making it more challenging to recognize the complete methods. However, our platform performed worst in the field of Nano Structure. This may because that the articles of Nano Structure include many complex and specialized terms in the subjects of biology, physics, chemistry, electronics, and metrology. Our platform still lacks the professional knowledge to learn the specific features.

The extracted top 10 problems of three fields are shown in Table 4, which reveal that Neural Networks focuses on the classification, prediction and recognition problems of data and images in the subject of Computer Science. Nano Structure covers a wide range, including physics, biology, chemistry, and so on, which focuses on the applications on the basic disciplines. Therefore, the extracted problems involve detection, analysis and prediction of energy, atom and medicine. The scope of Genetic Engineering is relatively narrow and is related to drug development, disease treatment, and biological manufacturing in the biomedical field. Table 5 shows the extracted top 10 methods. In the field of Neural Networks, they are mostly based on machine learning models, such as support vector machine, random forest, deep learning. The technologies in Nano Structure are specific instruments, such as microscope, spectrograph and ray. For Genetic Engineering, gene editing, manipulation and recombination are the three main techniques.

Entity Relation Prediction

By predicting the relations between problem and method, we construct the method-problem networks for different domains. As shown in Figure 5, the methods and problems which were separate in the articles of Neural Network are linked by relation prediction. The red dots refer to methods, and the blue dots refer to problems. Specifically, we can get more details from the abovementioned network. By setting the method X-Ray Diffraction (XRD) as a center, Figure 6 reveals that what problems are solved by XRD. They are Assisted Synthesis, Biomedical Application, Biosynthesis of Silver Nanoparticles and so on.

Hotspot Detection

Hotspots are the most popular research topics. We use the extracted keywords to pick out the hotspots in multiple domains. As a hotspot, the total number occurring in articles should be increased year by year or keeps a steady top order in last three years. According by this rule, Figure 7 shows the hotspots in the field of Neural Networks. They are distinct from the scientific entities recognized in section 3.1, which have no semantic type but reflect the popularity degree of terms. This paper introduced an innovative and intelligent platform IEKM-MD to extract information and mine knowledge from scientific articles in multiple domains. One contribution is providing a hybrid active learning strategy to solve the problem of annotated corpus scarcity in supervised learning model. Another contribution is designing an improved Translation embedding approach based on TransH model to optimize the performance of relation prediction. Three datasets in Neural Networks, Nano Structure and Genetic Engineering show that our platform is enable to achieve various knowledge services with a high accuracy in multiple domains.