Type-enhanced Inductive Knowledge Graph Completion

Abstract

Inductive knowledge graph completion has gained significant attention due to the dynamic nature of entities and facts in knowledge graphs (KGs). The goal of this task is to predict missing links between entities that are unseen during training. Graph neural networks (GNNs) have proven to be efective in handling this task. However, existing GNN-based methods overlook the type information of entities in KGs and thus may make incorrect predictions, which also limits the interpretability of the GNN-based models for KG completion. To address this limitation, we propose to incorporate type information into an existing GNN-based model for inductive KG completion. Experimental results show that our proposed approach is efective in improving the performance of inductive link prediction.

Keywords

• eol>Knowledge Graph
• Link Prediction
• Graph Neural Networks
• Type Information

1. Introduction

target relation. For example, the target triple (albert_einstein, worksAt, theory_of_relativity) may be predicted as true because there is evidence in the subgraph suggesting albert_einstein and theory_of_relativity are connected. However, they are linked by the relation develops not worksAt. By imposing type constraints and recognizing that theory_of_relativity is not a workplace, we can readily rectify this classification error. Secondly, the inclusion of type information can enhance the interpretability of models. Without the guidance type constraints, models may generate predictions that are clearly wrong and can be easily recognized by humans, as exempliifed by the erroneous prediction (albert_einstein, worksAt, theory_of_relativity). Introducing type information can help reduce such errors, thereby enhancing the model’s reliability.

In this paper, we propose to incorporate type information from KGs into an existing GNNbased model for inductive knowledge graph completion. This is achieved by a novel integration of type information in subgraph structures by prioritizing triples that are more likely to adhere to the type constraints. This is not straightforward since type information can be incomplete and the numbers of entities of diferent types can be diverse. To resolve these issues, we develop a method for inferring new entity type information from the existing entities and type hierarchies. Our model is the first attempt for incorporating type information into inductive KGC models. We note that while several works have been done on utilising type information for standard KGC [ 3 ][ 4 ], they cannot be directly applied to inductive KGC. Just as most inductive KGC settings, we only consider unseen entities, while relations and type information are seen in the training set. This is because in real-life applications, the relations and type information are usually more stable whereas entities may change. For instance, in e-commerce platforms, new products and users continually emerge, yet their types often remain consistent with the existing knowledge graph. Experimental results show that our proposed approach is efective in improving the performance of inductive link prediction, particularly in terms of ranking accuracy. The code and data used in our experiments are all available at https://github.com/Bohemianc/ISWC23-typed-inductive-LP.

2. Our Approach

In this section, we provide a detailed description of our solution that utilizes type information. First, we introduce the two forms of information we use, namely entity types and type hierarchies, and explain how we infer new entity types through type hierarchies. Second, we explain how we efectively integrate graph structures with type information, including details of joint training.

2.1. Type Mapping

We consider two forms of type information: (1) types of entities, such as (albert_einstein, rdf:type, Physicist) expressing that Einstein is a physicist, (2) and type hierarchies, such as (Physicist, rdfs:subClassOf, Scientist) expressing that physicists belong to the category of scientists. Let ℰ , , and ℛ denote the entity set, type set, and relation set in the KG, respectively. Entity type assertions are defined as (, rdf:type, ) with ∈ ℰ and ∈ , while type hierarchy assertions are defined as (1, rdfs:subClassOf, 2) with 1, 2 ∈ . Additionally, we refer to (1, , 2) with 1, 2 ∈ ℰ as an instance triple, and (1, , 2) with 1, 2 ∈ as a type triple.

In contrast to most existing works utilising type information [ 3 ] [ 4 ], we use explicit entity type assertions, which are more accurate than learnable type representations. However, this leads to two issues: incomplete entity type information and type imbalance (i.e., the numbers of entities of diferent types can be diverse). We believe that the impact of type imbalance on type representation is caused by overly specific entity types. For example, if there are much more mathematicians than physicists in the KG, the model will consider the type triple (Mathematician, develops, Theory) to be more likely than (Physicist, develops, Theory), while (Scientist, develops, Theory) is a more reasonable type triple that is not afected by type imbalance. To address these two issues, we use type hierarchies to explicitly infer new entity type assertions. Specifically, we apply the inference rule

(, rdf:type, ) ∧ (, rdfs:subClassOf, ) → (, rdf:type, ).

We recursively use this rule to supplement entity type assertions to address the incomplete entity type issue. To address the type imbalance issue, we use type hierarchy assertions to select the most general type among an entity’s multiple types, such as types Person and Location. Note that we do not consider the type Thing to prevent mapping all entities to this type.

2.2. Fusing Graph Structures with Type Information

Next, we introduce how to integrate graph structures with type information. For graph structures, we adopt the method proposed in RMPI [ 2 ], which is one of the state-of-art models in inductive KGC. This method transforms the sampled subgraph from the KG into another graph in which each node is an instance triple in the KG. Then, it applies a GNN to the transformed graph and obtain the final score using a linear layer. For type information, we assume that both relations and types during inference are already present during training, and thus we can represent relations and types using relation-specific and type-specific embeddings. Inspired by the translation-based principle in TransE [ 5 ], given a type triple (1, , 2), we expect that t1 + r ≈ t2, where t1, t2 and r ∈ R are embeddings of types or relations. Although we map entities to general types, the types of an entity are not necessarily unique. For entities with multiple types, we take the average score of corresponding type triples to obtain the likelihood of the type triple being true. By fusing graph structures and type information, our method prioritize those instance triples that are more likely to adhere to the type constraints. The score of an instance triple (, , ) is calculated as 2(, , ) = 1

∑︁ | ()| × | ()| 1∈ (),2∈ () score(, , ) = 1(, , ) + 2(, , ), −|| t1 + r − t2||2, (1) (2) where 1 is the score function of RMPI, 2 is the score function of type triples, and () is the set of types of an entity .

Following previous works, we adopt margin ranking loss for training. Specifically, we apply margin ranking loss to the final score score(, , ) and jointly train the two energy functions 1 and 2. An alternative training strategy is to separately train the energy functions,

Methods

GraIL

TACT CoMPILE RMPI-NE

Ours Methods

GraIL

TACT CoMPILE RMPI-NE

Ours

v1 similar to AutoETER [ 3 ]. However, optimizing 2 alone using margin ranking loss can lead to issues, as the corresponding type triple of a negative instance triple may not necessarily be false. For example, the type triple (Scientist, develops, Theory) of the negative triple (feynman, develops, theory_of_relativity) still holds.

3. Experiments

We conducted experiments on two benchmark knowledge graphs, FB15k-237 and NELL995, each with 4 versions split by GraIL [ 1 ]. Since the existing datasets lack type information, we obtained original entity types from external knowledge graphs or entity names and completed them as detailed in Section 2.1. Specifically, for FB15k-237, we map the anonymous entities in FB15k-237 to entities in DBpedia by sameAs.org1, and then retrieved entity types and type hierarchies through the meta-relations rdf:type and rdfs:subClassOf by querying DBpedia2. As for NELL995, its entity names inherently include a specific type of entities, and the associated website provides type hierarchies3. Our baselines include GraIL [ 1 ], TACT [ 6 ], CoMPILE [ 7 ] and RMPI-NE [ 2 ]. We select RMPI-NE, a variant of RMPI, as the representative of models proposed in [ 2 ] due to its superior performance than other variants. In our model, we also use RMPI-NE to calculate 1(, , ) in Equation 2. Our experiments aimed to demonstrate the efectiveness of our type-enhanced model in improving performance of inductive link prediction. 1http://sameas.org/store/freebase/ 2https://dbpedia.org/sparql 3http://rtw.ml.cmu.edu/resources/results/08m/NELL.08m.1115.ontology.csv.gz

Acknowledgments

This work was partially supported by the National Natural Science Foundation of China under grant 61976153.

References

[1] K. Teru , E. Denis , W. Hamilton , Inductive relation prediction by subgraph reasoning , in: International Conference on Machine Learning, PMLR , 2020 , pp. 9448 - 9457 .

[2] Y. Geng , J. Chen , W. Zhang , J. Z. Pan , M. Chen , H. Chen , S. Jiang , Relational message passing for fully inductive knowledge graph completion , arXiv preprint arXiv:2210.03994 ( 2022 ).

[3] G. Niu , B. Li , Y. Zhang , S. Pu, J. Li , Autoeter: Automated entity type representation for knowledge graph embedding , arXiv preprint arXiv: 2009 . 12030 ( 2020 ).

[4] J. Hao , M. Chen , W. Yu , Y. Sun , W. Wang , Universal representation learning of knowledge bases by jointly embedding instances and ontological concepts , in: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , 2019 , pp. 1709 - 1719 .

[5] A. Bordes , N. Usunier , A. Garcia-Duran , J. Weston , O. Yakhnenko , Translating embeddings for modeling multi-relational data , Advances in neural information processing systems 26 ( 2013 ).

[6] J. Chen , H. He , F. Wu , J. Wang , Topology-aware correlations between relations for inductive link prediction in knowledge graphs , in: Proceedings of the AAAI Conference on Artificial Intelligence , volume 35 , 2021 , pp. 6271 - 6278 .

[7] S. Mai , S. Zheng , Y. Yang , H. Hu , Communicative message passing for inductive relation reasoning , in: Proceedings of the AAAI Conference on Artificial Intelligence , volume 35 , 2021 , pp. 4294 - 4302 .