Acquiring insights and understanding from historical pandemics is crucial for reducing the likelihood of their recurrence. The utilization of knowledge graphs stands as an essential tool for researchers, with knowledge inference emerging as a prominent task within these graphs to deduce previously unidentified connections between entities. This study endeavors to construct a knowledge graph centered on pandemic research and to evaluate the eficacy of various mainstream methodologies in the context of biomedical association inference. Our findings indicate that techniques for graph representation hold significant promise in executing these tasks and heterogeneous graph representation techniques demonstrate high predicting accuracy. Nonetheless, the advancement in this area of research necessitates more refined experimental designs and the adoption of more adaptive learning strategies.
Biomedical entity association inference is a long-term task
for scientific researchers and industry practitioners to
understand the relationships between biomedical entities and
propose first-hand literature-based evidence for further
investigations [
Knowledge graphs, which are extensively used to depict
intricate data relationships, serve as the foundation for
analyzing and inferring associations [
In this study, we developed a knowledge graph from scholarly articles on SARS, MERS, and COVID-19, comprising 9,142 nodes and 81,707 connections. We conducted a validation test to assess how well various mainstream techniques could predict relationships within this graph. By masking 10% of the connections of each type, we applied five diferent methods to the masked graph to identify the hidden connections from an equal mix of randomly inserted non-existent connections. The findings revealed the diverse efectiveness of these methods in identifying the obscured connections, with HetGNN proven as the most efective. Nonetheless, the lfexibility and applicability of diferent graph representation methods across varied contexts need enhancement. This research illustrates the application of multiple prominent methods in deducing associations in knowledge graphs and verifies the precision of these methods.
The following of this paper is organized as follows: We introduced the pandemic knowledge graphs and examined methods in the section Data and Method, followed by Experimental Settings and Results. We concluded the study and anticipated some future directions in the section of Discussion and Conclusions.
The integrative Biomedical Knowledge Hub (iBKH) is a
knowledge graph dataset that curates the associations of
11 categories of biomedical entities from 17 publicly
available data sources [
The pandemic graph is denoted as = (, ), and
= {, , }
= {, , , , , }
(1)
(2)
where , , and respectively represent the node
set of diseases, drugs and genes. (, ∈ {, , })
denotes the edge set of associations between nodes of types
and . Entity association inference on this pandemic graph
aims to predict emerging associations between nodes in
that have not yet appeared in .
• Random Walk with Restart (RWR) is a commonly
used method for inferring relationships within
graphs, particularly in the biomedical field. It
models a random walking process that begins at node
and calculates the likelihood of reaching node as
a measure of relevance between nodes and . To
avoid the walk from becoming trapped in local areas,
it introduces a restart probability , which allows
the walk to restart from node at each step, thereby
ensuring broader exploration of the graph.
• Resource allocation (RA) [
The setup for the experiment is detailed in Figure 1. The objective was to assess the eficacy of various algorithms in predicting associations between biomedical entities. To this end, a validation experiment was structured in the following manner: From each category of edges, denoted as where , belong to the set , , (representing disease, drug-gene, and gene respectively), 10% of the edges were randomly selected and removed. The resulting graph, with these edges removed, was labeled as = (, ). The edges that were removed are represented by = |, ∈ , , , and these were considered the ’true’ associations for the purposes of this experiment. In addition to this, an equivalent number of node pairs, which were not connected by edges in the original graph , were randomly chosen. These pairs are denoted by = |, ∈ , , , ∩ = ∅, and they were defined as the negative sample set for this study. This methodical approach enabled a balanced evaluation, comparing the algorithms’ abilities to correctly infer both existing and non-existing associations, thereby providing a comprehensive understanding of their performance in the context of biomedical entity association inference.
Subsequently, each candidate algorithm was applied to the modified graph to ascertain the likelihood of edge formation between every pair of nodes within both and . In the cases of the Random Walk with Restart (RWR) and resource allocation algorithms, this procedure involved computing the random walk probability and the resource allocation score, respectively, for each node pair. Conversely, for the three graph representation techniques, the process entailed converting every node in the set into embedding vectors. The representation for edges was then determined through an average pooling strategy, which involves aggregating the features of node embeddings to form a single representation for each edge.
Following the generation of these probabilities or representations, the combined dataset of and was divided, with 80% allocated for training and the remaining 20% for testing. This division was employed to train a logistic regression classifier, the purpose of which was to predict the likelihood of edge formation between node pairs in the test set. The predictions made by the logistic regression model were then used to calculate the Area Under the Curve (AUC) metric for each method. By focusing exclusively on the test data, which comprised 20% of the total dataset, a standardized evaluation criterion was established. This approach allowed for a fair comparison of the five candidate methods, with the AUC metric serving as a measure of each method’s ability to accurately classify node pairs as either connected or not connected, based on the generated classification probabilities.
Table 2 presents the AUC scores for the five candidate methods. It is noted that HeCo needs a metapath definition to function, and a gene-based metapath was chosen for this purpose. Consequently, HeCo’s evaluation was limited to gene-related associations. It was found that HetGNN outperformed others in recovering the removed links.Compared to RWR and RA, the three graph representation methods demonstrated better accuracy in identifying connections. Yet, their advantage is not definitive because they utilize a supervised learning approach, requiring both positive and negative samples to train a classifier, whereas RWR and RA can be applied directly to the existing graph structure without any pre-existing knowledge of it.
From the perspective of edge types, the analysis of genedrug and drug-drug connections showed superior outcomes. Importantly, both RWR and RA displayed similar levels of efectiveness as graph representation techniques in the task of deducing disease-disease associations. This suggests that inferring disease similarities might be distinct from other tasks, meriting additional investigation.
Among the graph representation strategies, two methods tailored for heterogeneous networks achieved superior AUC scores over Node2Vec. This superiority results from their training mechanisms being specifically designed for heterogeneous networks, as seen in this research and commonly in biomedical entity graphs. These methods incorporate the significance of node types into the computation, employing either type-specific or metapath-based aggregation strategies for information. While this heterogeneity-focused approach is beneficial, it limits the model’s applicability and increases the cost of adaptation. Changes in the heterogeneous graph’s structure necessitate adjustments to HetGNN’s data inputs and HeCo’s metapaths, along with significant methodological revisions. Additionally, HeCo’s performance is influenced by the setting of a positive sample threshold and the definition of metapaths, which vary per case and afects the outcome significantly. Node2Vec, in contrast, ofers a more generalized solution applicable to a wide range of graph types.
In conclusion, while heterogeneous graph representation methods hold promise for deducing relationships within pandemic knowledge graphs, enhancing their flexibility and general applicability remains a challenge.
This study explores the performance of diferent methods of
association inference and provides insights into the
potential of graph representation methods. Despite some existing
entity-relationship summarization tools like PubTator 3 [
We anticipated the following future directions aligning with some limitations of the current study: 1) This study ofered some preliminary understandings on selected baselines of graph representation learning in inferring the pandemic knowledge graph, but further customized redevelopment based on the unique features of the pandemic knowledge graph to enhance its performance might be beneficial. 2) Investigating the scientific community of a pandemic and its collaborative patterns will bring insights to analyze the societal context of a pandemic crisis and provide evidence-based decision support in terms of science policy, public health, and public administration.
This work was supported by the Commonwealth Scientific and Industrial Research Organization (CSIRO), Australia, in conjunction with the National Science Foundation (NSF) of the United States, under CSIRO-NSF #2303037.