Link prediction task intends to complete Knowledge Graphs (KGs) which are always far from complete. Textual descriptions of entities in KG provide additional information that may not be explicitly represented in the structured part of the KG. Current methods aim to learn the representation of KG and predict missing links by utilizing structured and textual information. In this poster, we propose a novel rerank method that introduces the natural language inference task to leverage textual information of entities in a diferent way. The experiment demonstrates that our rerank method improves the quality of link prediction.
Various Knowledge Graphs (KGs) such as Freebase and ConceptNet have been
published to share linked data and have been crucial for many tasks.
However, according to the Open World Assumption, KGs are never complete. Due
to this fact, diferent KG representation learning (RL) models map KGs to a
low dimensional vector space and predict missing facts. TransE [
However, these translation-based RL methods only utilize the structural
information of KG and ignore the rich information contained in entity
descriptions. Fig. 1 presents an example of a tirple with entity descriptions sampled
Copyright © 2021 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
from Freebase. Therefore, some methods such as DKRL [
Motivated by [
( Dominican Republic, form of government, Republic )
The Dominican Republic is a nation on the island of Hispaniola, part of the Greater Antilles archipelago …
A republic is a form of government
in which power is held by the people and representatives they
elect …
We only introduce the TransE model because these translation-based RL models
are similar. Given entity set E, relationship set R and triple set S in which each
triple (h, r, t) consist of two entities h, t and a relationship r, the task of the
model is to learn embeddings of entities and relationships. TransE[
d(h + r, t) = ∥h + r − t∥22 TransE uses margin-based ranking loss:
LRL = ∑
∑ (h,r,t)∈S (h′,r,t′)∈S′
[γ + d(h + r, t) − d (h′ + r, t′)]+ where γ is a margin hyperparameter, [n]+ represents the positive part of a number n, and the negative triple set S′ consists of corrupted triples which replace the head or tail entity with a random entity. (1) (2)
Given a dataset D = {(s1, s2)i , yi}iN=1, Natural Language Inference (NLI) intents to learn a function fNLI (s1, s2) → {E, N, C} which predict the relationship of input pair (s1, s2). The input (s1, s2) are two natural language sentences and denote the premise and hypothesis respectively. The label y is one of three classes {entailment, neutral, contradiction} which represent entailment, natural and contradiction relationships between premise and hypothesis. 2.3
To leverage text description of entities through NLI, we need to generate
sequence pair and relevant label from triples S. We transform three classes
{entailment, neutral, contradiction} to two classes {entailment, contradiction} for
N
the consistency of datasets. We construct NLI dataset D = {(s1, s2)i , yi}i=1 from
triple datasets S and S′. For each triple (h, r, t) ∈ S, we construct {(s1, s2) , y}
by mapping two entities (h, t) to relevant textual descriptions (htext, ttext), where
s1 = [htext; ttext], s2 = [h; r; t] and y = 1. If triple (h, r, t) ∈ S′ , y = 0. In order to
ensure the quality of generated dataset, we construct the negative triple dataset
S′ by transforming the first top negative sample of link prediciton by TransE.
We use Bert [
LNLI = N1 ∑ − [yi · log (pi) + (1 − yi) · log (1 − pi)] where pi is the classification probability of sequence pair with Bert. 2.4
First, given a triple (h, r, t), we use all entites to replace head or tail entity and calculate scores of all triples called scoreT ransE . Secondly, we sort triples depended on scores and get the top 10 triples. Then, we get their scoreBert which is the classification probability computed by Bert. New scores are then computed as:
scorererank = scoreT ransE − λ ∗ scoreBert and the 10 triples are reranked according to scorererank. (3) (4) 3
We verify our rerank method on two datasets, namely, FB15k and FB15k-237.
To confirm that all entities in datasets have descriptions, we follow the process
in DKRL [
By Table 1, we show that our rerank method achieves higher scores of all metrics compared to original TransE and TransH models on two datasets. Besides, we even achieve 43% higher score of Hits@1 on FB15k, which demonstrates the usability of link prediction by our model. We can conclude the efectiveness of our rerank method.
Using text descriptions of entities has been proved to be an valid way to improve the quality of link prediction. In this poster, we propose a rerank method for link prediction which is a diferent way to leverage the text information of entities. Experiments demonstrate that our approach is useful and promising. In future work, we are interested in extending our rerank method for more KG Embedding models.