Scientific research collaboration has always been an important research content of information science, and collaboration prediction is an important issue in personalized information services. This article constructs an author-centered heterogeneous information fusion schema, and uses causal analysis to quantitatively study the influence of same institutions, co-word and citation on collaboration, compares the effects of different network embedding algorithms in collaboration prediction, and built heterogeneous information fused network embedding model for collaboration prediction. Take the field of stem cells as an empirical case, experiments show that the matrix decomposition based network embedding algorithms(like NetMF) are balance of performance and accuracy. Institutions, keywords and citations can improve the prediction effect of collaboration, and (same institutions > citations > co -words) among them. Multiple features fusion models are generally better than single information fusion, and the model of (collaboration + same institution + citations) performs outstanding in collaboration prediction.
Since the birth of information science, scientific research collaboration has become an important research topic. The prediction of collaboration has important theoretical and practical significance for the analysis of S&T trends and the recommendation of personalized service information. Collaboration prediction is also a very challenging task. Scholars in different fields have invested in this topic. Most computer scientists design more and more advanced and complex network representation learning algorithms, and find ways to incorporate different types of information such as text into representation learning. Intelligence experts pay more attention to the application of these algorithms for collaborative recommendation
Newman[1] was the first to introduce the network analysis into research collaboration. Liben-Nowell and Kleinberg [2] put forward the problem of link prediction in social networks, they gave some similarity measurements based on network structure, and applied two series of index methods based on nodes and paths, empirical analysis is carried out in the collaboration network of authors in five fields of physics. Since then, quit a lot of work focused on improve the indicators in this article. In the book "Link Prediction", Lv Linyuan introduced a series of similarity-based link prediction indicators, such as common neighbor index (CN), cosine similarity, Jaccard Indicators, Adamic-Adar (AA) indicators, resource allocation (RA) indicators, LHN-I indicators and others based on the local information of nodes, as well as local path indicators (LP), Katz indicators, restart random walks, etc. based on the information of edges or paths[3]. Intelligence experts has conducted a series of empirical studies based on these indicators, such as R Guns and R Rousseau [4] ,they constructed a weighted collaboration network in the field of malaria and tuberculosis to carry out cooperative prediction and recommendation work. Yan E et al. [5] used the papers of 59 journals in the field of library and information as an empirical case to compare the prediction results under various indicators. Shan Songyan et al. [6] summarized and reviewed the author similarity algorithm for cooperative prediction. In addition, cooperative prediction also has methods based on maximum likelihood estimation, methods based on probability graph models, etc. The latter two approaches are based on statistical ideas and have achieved certain effects on specific networks. The computational complexity of those method is too high, and it is difficult to implement on large-scale networks.
In response to the large-scale sparse network prediction problems encountered in actual research, researchers continue to propose new methods to learn a low-dimensional dense representation of the network. Then use low-dimensional representation for structural prediction work. For example, Zhang Jinzhu et al[7] introduced the method of network representation learning into the collaboration prediction. In this paper, the LINE model is used to construct the vector representation of author, and the cosine similarity is used to measure the possibility of collaboration. Yu Chuanming et al[8] studied the application of network representation learning methods in collaborative recommendation, proposed an integrated recommendation model, and conducted empirical analysis in the financial field.
Moreover, it must be noted that collaboration is affected by many complicated factors. In addition to the network structure, collaboration prediction should also use rich heterogeneous structure information. Wang Zhibing et al. [9] merged the attribute characteristics such as the node mechanism into the similarity index of the network structure, and carried out collaboration prediction. Liu Ping et al. [10] constructed an LDA-based author interest model based on the community division, and then analyzed the author ’ s relevant literature to achieve the purpose of scientific research collaboration recommendation; Yu Chuanming et al. [11] constructed the collaboration network in the financial field and adopted a link prediction method based on feature fusion of individuals, institutions and regions. Lin Yuan et al. [12] combined the heterogeneous information of scholars, institutions, and keywords to construct a scientific research collaboration network, and then used node2vec to express learning methods for cooperative prediction. Zhang Xin et al. [13] proposed a scientific research collaboration prediction method that combines network representation learning and author's topic characteristics .On the basis of these studies, this paper constructs a cooperative prediction method of feature fusion. 2 2.1
The research framework of this paper is shown in Figure 1. It can be roughly divided into 6 stages, namely (1) data collection -> (2) exploratory data analysis and network construction -> (3) the impact of heterogeneous information on scientific research collaboration -> (4) embedding algorithms evaluation for link prediction -> (5) The construction of heterogeneous features fused network embedding approaches and (6) Results and analysis. Retrieve the documents to be analyzed from the Web of Sciences database, and then conduct exploratory analysis on the collaboration relationship among them, and construct a collaboration network, author-institution network, author-keyword network, author-citation network. Later, for the same institutions, shared keywords and citation relationships, the framework of causal analysis was used to study their impact on collaboration. Then, we discuss the performance and efficiency of DeepWalk, LINE, HOPE, Node2vec, SDNE, NetMF and ProNE and other models for collaboration predictions. Finally, we integrate the Institution(I), Keyword(K) and Citation(C) features to network embedding models and use these schema to carry out actual network prediction task. Heterogeneous information fusion is abbreviated as data fusion, which refers to the comprehensive analysis of different types of information sources or relational data through a specific method, and finally all the information can be used to jointly reveal the characteristics of the research object, making up for the single data type and the single relation type to reveal the research field insufficient associations between entities to obtain more comprehensive and objective measurement results (Xu Haiyun et al. [14]).
Hua Berlin [15] put forward the fusion theory as one of the main methodologies of information science , and emphasized the important of data fusion, information fusion and knowledge fusion in information science. Later, he further discussed the influence of data fusion on intelligence. The importance of fusion of different types of multiple source information is analyzed. He also systematically explained the relevant theories and applications of information fusion from the perspective of information fusion's representation process, technical algorithms and models. Morris et al. [16] gave an overview of common measurement entities in scientific and technological literature, mainly including documents themselves, references, journals where documents are located, authors of documents, journals where references are located, authors of reference documents, subject headings etc. Xu Haiyun et al. [14] reviewed the multiple source data fusion methods in Scientometrics. The document-centered data fusion meta-path model given in the paper is shown in Figure 2(a). This article takes the author's collaboration as the research object, and studies the influence of the same institution, the same keywords, and the citation relationship on scientific research collaboration. On the basis of Figure 2(a), an author-centered scientific research collaboration network is constructed, as shown in Figure 2(b). (a) Document-centered schema (b) Author-centered schema
Fig. 2. Heterogeneous Information Fusion Schema
collaboratio n Institution Keyword Citation Author-Document Bipartite Network Author Citation Network
Document Citation Network
Table 1 shows several methods for constructing heterogeneous network information. The first column represents the fused heterogeneous information, the second column represents the feature network that can be directly extracted from the article, and the third column represents the mapping of the second type of network to the author dimension The formed network.
(1) Author collaboration network. The author collaboration network can be extracted directly from the author field of the article. The nodes in the network represent authors, the edges in the network represent collaboration relationships, and the weight of the edges represents the frequency of collaboration.
(2) The same institution network, extract the author institution bipartite network from the author-institution (C1) field of the article. An institution can contain multiple authors, and at the same time, an author can work part-time in multiple institutions, which can be mapped into Authors are in the same organization network. The nodes in the network represent authors. If two authors have the same organization to which they belong, the network is connected.
(3) The author co-word network, according to the downloaded documents, we can construct an article-author network, article-keyword network, and then form an author-keyword bipartite network. The weight d(a,k) in the network indicates that the the frequency of author a uses keywords k (number of articles), (a) represents the neighbor node of node a, that is, the keyword used by a. According to this network, we can deduce the author co-word network. The nodes in this network represent the author. The edge weight w(a,b) between author a and b is calculated by formula (1), D represents the total number of documents, and d(k) represents the frequency of keyword k.
w(a, b) min(d (a, k ), d (b, k )) log (1) k(a)(b) (4) Author citation network, based on the downloaded documents, an author-article bipartite network can be constructed (the article written by author a in this network is denoted as (a) ), an article citation network, and then an author’s citation network is formed. The weight c(a,b) in this network is the number of articles of author b cited in all articles of author a.
c(a, b) (d , c) (2)
d(a) c(b) if document d cite document c， (d , c) 1 ，Otherwise (d , c) 0 .Unlike the D d (k ) previous two networks, the author cited network is a directed network.
In this way, we have constructed several heterogeneous information networks. However, the influence of different heterogeneous information on collaboration may be different. In order to measure the difference of this influence, we adopt the paradigm of causal reasoning. ACE (Average Casual Effect) is used to measure the influence of treatment variable Y (same institution, co-word, citation) to variable X (collaboration).
ACE(Y X ) E( X | Y ) E( X |~ Y ) (3) The larger the ACE, the more obvious the causal effect of Y on X. 2.3
Network embedding is currently a hot method in the field of cooperative prediction. Scholars continue to incorporate new ideas and methods into network representation learning, and the accuracy and efficiency of the algorithm continue to improve. This article compares seven well-known and commonly used network representation learning algorithms and evaluates their accuracy and performance in scientific research collaboration prediction.
Given a network G(V,E), V is the set of vertices, E is the set of edges, |V|=N,|E|=M, the adjacency matrix of the graph is W.
(1) DeepWalk, influenced by the well-known word2vec, Perozzi et al. proposed the DeepWalk [17] in KDD2014. The model uses random walks to generate vertex sequences, and each node sequence is analogous to a sentence in the language. Each node pair is analogous to a word, and the Skip-gram method is used for learning and training, and the vector representation of the node is obtained.
(2) LINE (Tang Jian et al., 2015) [18] defines the first-order similarity relationship (1st order proximity) and the second-order proximity relationship (2nd order proximity) in the graph in the algorithm. The distance is closer. The algorithm defines first-order and second-order optimization goals to characterize this relationship.
(3) Node2vec is a work published on KDD2016 by Grover A et al. [
(4) SDNE (Wang D et al., 2016) [
(5) The HOPE [
(6) NetMF, Tang J team unified deepwalk, line, node2vec and other algorithms
into the framework of matrix decomposition, and proposed a NetMf representation
learning method that directly decomposes the target network [
(7) ProNE, Jie Zhang from the team of Professor Tang Jie of Tsinghua
University proposed a fast and scalable large-scale network representation learning
algorithm ProNE[
The network embedding approaches and graph neural networks are moving from
theory to application. In 2018, the Alimama team open sourced Euler, a distributed
graph deep learning tool, DeepMind open sourced the graph_nets graph network tool,
New York University researchers open sourced the graph neural network learning
framework DGL, and in 2019 Facebook open sourced PyTorch-based graph
representation learning and neural The network framework PyTorch-BigGraph [
In this paper, we use those network embedding for collaboration prediction. The
collaboration prediction problem is transformed into a binary classification problem
with/without links. According to the specific ratio, the edge set is divided into training
set and test set respectively, then taking the training set as the baseline, we take the
edges in the test set as positive samples, randomly generate the same number of
negative samples to evaluate the model. We use the AUC, ROC_AUC and F1_Score
indicators to evaluate the accuracy of the model. The AUC indicator is the most
commonly used indicator to evaluate the link prediction problem. Provided the rank of
all non-observed links, the AUC value can be interpreted as the probability that a
randomly chosen missing link is given a higher score than a randomly chosen
nonexistent link [
The method of fusing multiple features is shown in Figure 3. It extracts the basic collaboration relationship, the same organization relationship, the co-word relationship of articles, and the citation relationship between authors from the document collection, and constructs the collaboration network, the same organization network, the co-word network, and the citation network. 3
Stem cell and regenerative medicine research has brought revolutionary changes to the treatment of cancer and other diseases. It has been selected as one of the top ten scientific and technological advances in the US "Science" magazine for 9 times. The project has also laid out related projects many times, so the author selects the field of stem cells for empirical research. Search in ISI Web of Knowledge with the search formula (TI=Stem Cells). The search time was May 2019. The search yielded 433 469 articles. The number of articles in different years in the search results is shown in Figure 4, which shows that the number of articles is presented The trend of slow growth to rapid growth and then to saturation growth may have declined in 2019 due to incomplete data collection.
Analyzed from the ISI database, DE author keywords, C1 field represents the author institution, the citation relationship between articles is obtained from the CR field, and the author and institution information is split from the author institution field. The institution information only intercepts information such as university hospitals.
After splitting, 1,682,654 authors were obtained, and 1,461,721 authors were merged. More than 40,000 authors were selected as the first author to publish articles. Considering calculation performance, we selected 5,403 of these authors who have collaborated with other authors more than 2 times. The total number of collaboration between these authors is 4,818, forming Table 3 shows the basic statistical characteristics of the benchmark cooperative network by year. #Nodes
#Edges 65 402 529
Predictability is an important research problem in link prediction. The
predictability of the network represents the upper limit of prediction. Random
networks are completely unpredictable. Any link prediction algorithm will not get
better results on completely random networks. The predictability of the network is
related to the characteristics and evolution of the network itself. Two articles by
Newman et al [
Figure 6 shows the eigenvalue distribution of the critical matrix of the studied network. The largest eigenvalue is about 23.37. There are obvious gaps on the right and left sides of the figure, and the network will have better predictability. 3.2
In this part, we use the method introduced in section 2.2 to compare the effects of the same institution, co-word, and citation on the probability of collaboration by year.
First, we studied whether the same institution has a causal effect on scientific research collaboration in a large network composed of the first authors of all cooperative articles, and calculated separately the same institution-there is collaboration, the same institution-no collaboration, and different institutions-the side of collaboration. The number, and then subtract the first few numbers from all possible edges to get the frequency of different organizations-no collaboration. The results are shown in Table 4, suppose that event X: there is collaboration between authors, and Y: the authors have the same organization.
Table. 4. Same Institution and its impact of collaboration in the big network
X ~X Y 16205 1908127 ~Y 24937 984907682
P(X|Y)=16205/(16205+1908127)=8.421e-3
P(X|~Y)=24937/(24937+984907682)=2.5318e-5 The average causal effect of event Y on event X:
ACE(Y—>X)=P(X|Y)-P(X|~Y)=8.396e-3 The probability of collaboration is increased (PI) by 331.62.
Next, we disassemble the collaboration network of 5403 nodes and 4818 edges according to the year to discuss the causal effect of scientific research collaboration with the institution each year. The results are shown in Table 5.
Table.5. The impact of the same institution on collaboration by year Year 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 collaboration, and the probability of scientific research collaboration with the same institution is increased by 200-300 times.
This part discusses whether the authors have common keywords that affect scientific research collaboration. Let event X: there is collaboration between authors, and Y: there are common keywords used by authors. The method similar to section 3.2.1 is used for annual calculation. There is a causal effect of common use of keywords on scientific research collaboration. The results are shown in Table 6.
Table. 6. The impact of the co-word on collaboration by year Year 2007 2008 2009 2010 2011
X&Y 0 0 3 44 115 2012 145 411 1537 542353 7.5724E-04 8.6207E-02 112.84 2013 213 471 2518 783683 6.0065E-04 7.7993E-02 128.85 2014 259 483 4132 912461 5.2906E-04 5.8984E-02 110.49 2015 312 586 5165 1140792 5.1341E-04 5.6965E-02 109.95 2016 325 416 4527 951768 4.3689E-04 6.6983E-02 152.32 2017 294 349 5209 786059 4.4379E-04 5.3425E-02 119.38 2018 320 237 5521 546748 4.3328E-04 5.4785E-02 125.44 2019 126 70 728 75712 9.2370E-04 1.4754E-01 158.73
It can be seen from Figure 6 that for each time slice of the collaboration network, the causal effect of citations on collaboration is strong, and the citations increase the probability on collaboration by about 100 times. Not as obvious as the causal effect of the same institution, but obviously stronger than the effect of keyword. 3.3
Divide the data set into training set and test set according to the ratio of 80%-20%, 60%-40%, and 40%-60%, and discuss the performance of the seven algorithms introduced in section 2.3. Each algorithm takes the same embedding dimension, and the 4 numbers in each cell represent the (ROC_AUC, AUC, F1_Score, Run time) introduced in Section 2.3.
Table. 8. Performance and efficiency of embedding algorithms selection. The accuracy of matrix factorization models such as ProNE and NetMF is not much different from the LINE model, but the time efficiency is improved by hundreds of times. Especially for the NetMF model, the accuracy is very small and the time efficiency is very high. Therefore, the following experiments choose NetMF as the network representation learning model. 3.4
In this part, we select the collaboration data of a certain year as the positive examples of the test data, randomly generate the same number of negative examples as the test positive examples, and select the collaboration data, institutional data, and author co-word data before that year (excluding the year). The cross-citation data is fused as a training set. Several types of feature fusion methods discussed in Section 2.4 are used for experiments. Table 9 shows the results of single feature fusion, Table 10 shows the results of multiple features fusion, and the three numbers in the cell in the table indicate (ROC_AUC, F1_Score, AUC), the best result in each row is expressed in bold.
Year 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019
Combining Table 9 and Table 10, it can be clearly seen that (1) Almost all feature fusion cooperative prediction methods are larger than the original network in ROC_AUC, F1_Score, and AUC, indicating that the feature fusion method can improve the accuracy of cooperative prediction. (2) Same institution (I)> reference relationship (K)> same keyword (C), which is exactly the same as the causal effect sequence of the several types of relationships discussed in Section 3.2. (3) The accuracy comparison of multiple features fusion methods is generally I+C> I+K+C> I+K> K+C. In the case of simple network nodes and relationships, the I+K+C three-feature fusion method is better than I+C two features. Multiple features can bring more relationships and improve the prediction effect. There are relatively many data nodes later, and the prediction result is close to the upper limit of the predictability of the network. Adding keyword co-occurrence features may cause more confusion in the network due to frequently occurring words, and the prediction effect will not be further improved. 3.5
We randomly select scientific researchers and predict collaborators for him/her. In this example, we selected the researchers Lin Mingyan of Albert Einstein Coll Med, and used the NetMF of Raw+I+C feature fusion with better results in the experiment in Section 3.4. Table 11 lists the top 20 authors with the highest probability of collaboration in the future.
Table. 11.
Author
Institution sim
Author
Institution.
Sim Zheng, Albert Einstein Coll 0.9977 Delahaye, Albert Einstein 0.9783 Deyou Med Fabien Coll Med Pedrosa, Albert Einstein Coll 0.9967 Rockowitz, Albert Einstein 0.9768 Erika Med Shira Coll Med Chen, Jian Albert Einstein Coll 0.9965 Wijetunga, N. Albert Einstein 0.9643
Med Ari Coll Med Zhao, Dejian Albert Einstein Coll 0.9937 Pal, Rajarshi Manipal Univ 0.9590
Med Branch Campus Wang, Ping Albert Einstein Coll 0.9928 Carromeu, Univ Calif San 0.9240
Med Cassiano Diego Xue, E. Albert Einstein Coll 0.9924 Marchetto, Salk Inst Biol 0.9225
Med Maria C. N. Studies Sharma, V. Albert Einstein Coll 0.9924 Zhou, Li Albert Einstein 0.9148 P. Med Coll Med Abrajano, Albert Einstein Coll 0.9904 Jaffe, Andrew Lieber Inst 0.8058 Joseph J. Med E. Brain Dev Guo, Xingyi Albert Einstein Coll 0.9891 Lei, Mingxing Univ So Calif 0.7802
Med Qureshi, Albert Einstein Coll 0.9851 Will, Britta Albert Einstein 0.7758 Irfan A. Med Coll Med
The first few authors in the results seem to be the first few authors who have worked closely with the author, and the chances of continuing to cooperate in the future are also very high. In the results of the collaboration recommendation, the proportion of the same institution as the author is very large, accounting for 75%, which is also consistent the law of scientific research collaboration. 4
This paper constructs the collaboration prediction method based on heterogeneous information fused network embedding, and conducts an empirical analysis in the field of stem cells.
(1) Construct a author-centered heterogeneous information fusion schema, based on information fusion theory. The predictability of the scientific research collaboration network and the effects of institutions, co-word, and citation information on collaboration are discussed. Experiments show that the they have an impact on collaboration. The average causal effect analysis shows that the influence order of the three factors is as follows(same institution> citation> keywords).
(2) The accuracy and efficiency of the network representation learning methods for collaboration are discussed. Experiments show that the accuracy and computational efficiency of the comprehensive method, and the graph representation learning method based on the new matrix factorization (like NetMF) has achieved good results.
(3) Construct a prediction method for scientific research collaboration based on heterogeneous information fusion, and conduct empirical analysis in a yearly network. Experiments show that the method of multiple features fusion can greatly improve the accuracy of collaboration prediction. In terms of feature combination, The combination of the same institution + citation relationship has achieved outstanding results.
Future research will build some more complicated causal diagram under the author-centered of information fusion framework, explore the causal effects of feature combinations; explore more detailed methods for selecting relationships within features, and continuously explore relationships in various information, and improve the identification effect. Expand the framework of information fusion, and introduce other invisible features such as research fields, research topics, and writing styles into the framework of collaboration prediction, and continuously enrich the connotation of the methods. 10. 11. 12. 13. 14. 15. 16. 17. 18.