Fact validation in a knowledge graph is a task to determine whether a given fact (subject, predicate, object) should appear in the knowledge graph. In this paper, we have described our approach for the fact validation task in the context of the Semantic Web Challenge 2019. We used embedding features with machine learning to predict facts that were missing from the knowledge graph. The embedding features were generated applying a knowledge graph method known as the RDF2Vec method on the knowledge graph with only positive statements. To improve our machine learning model, we added the test facts that we could validate via the public sources into the positive knowledge graph. We trained a Random Forest classi er on the training data (positive and negative statements) plus the veri ed test statements and made predictions for test data.
Knowledge graphs are currently among the most prominent implementations
of Semantic Web technologies. The task of fact checking in knowledge graphs,
which is to decide whether a fact t is missing from a given a knowledge graph
G, is among the cornerstones of knowledge base management. The veri ed facts
can be used to (1) incomplete knowledge graphs re ning (2) knowledge graphs
violation detection, (3) improve the quality of knowledge search, and (4) multiple
knowledge graphs integration [
The core dataset consists of a graph of entities (drugs, diseases and products) and information linking these entities. The dataset is created by extracting information from a well-known source and identifying links between entities. The dataset contains both training and test set parts in which the training data was made available for building a system to make prediction on test data. Both the training and testing sets consist of 25k examples with positive and negative statements, equally distributed among each of the following ve properties: { http://dice-research.org/ontology/drugbank/interactsWith { http://dice-research.org/ontology/drugbank/hasCommonIndication { http://dice-research.org/ontology/drugbank/hasSameState { http://dice-research.org/ontology/drugbank/hasIndication { http://dice-research.org/ontology/drugbank/hasCommonProducer
While the challenge organizers generated positive statements by identifying the entities for which the proposed properties hold, they generated the negative statements by replacing the entities in the positive statements such that the generated triples are false or invalid. 3
We used embedding features to train our machine learning model to predict trust scores for test facts as described in Figure 1. In order to learn embedding features, the training KG, given in the form of rei ed statements, was converted to the positive explicit statements (eg. < drug; hasIndication; disease > ). We trained our classi er on the training data (positive and negative statements) plus the veri ed test statements and made predictions for test data.
We added the veri ed facts to the positive knowledge graph to generate a better embedding feature vector. We used the external resource "DrugBank" to check if the DrugBank relations might hold for the test facts. This can be seen as the enrichment of the training knowledge graph. While enriching with the DrugBank, only the properties that are relevant with the challenge dataset were included. The properties extracted from Drugbank XML (v5.1.1) are: { indication { state { drug-interactions { packagers
Fact Validation with Knowledge Graph Embeddings
To validate test facts, we link the challenge data to the Drugbank dataset by mapping the drug and disease entities to DrugBank drugs and Human Disease Ontology (DO) diseases respectively. The challenge use the same unique identiers with Drugbank for drugs. In order to link the challenge disease entities, each text in the indication section in Drugbank was annotated with Human Disease Ontology using BioPortal API.
After disease annotations were obtained for both training and test datasets drugs, the normalized Levenshtein similarity was computed between disease names to match the annotated diseases (DO) with the challenges diseases. A test fact will be considered as veri ed if the proposed relation between the identi ed entities holds in the DrugBank.
For feature learning, we need a proper representation of the entities in the
knowledge graph that re ects their features. Here, we used the RDF2Vec
approach [
The Fig. 2 shows the work ow of generating the embeddings. After that, a Random Forest classi er was trained on the training set plus the veri ed statements. A number of estimators of 200 was used for the Random Forest classi er and the remaining parameters were left as default. To represent the feature vector of a statement, we concatenated embedding vectors of the subject and object entities, and a numerical value encoding predicates between the entities. The nal model was used to predict a probability for each statement in the test set. The predictions were submitted to the challenge website and reported an AUC of 0.99971976. The reported AUC was the highest obtained score among all the participants in the challenge as provided through the leader board on the challenge website 3. The same method was also applied on the original dataset without performing the enrichment using Drugbank and reported an AUC of 0.9926. From these results, a conclusion can be drawn that the major contributor to the performance is the proposed embedding-based method over enriching the data with an external resource.
Acknowledgments This work was supported by funding from King Abdullah University of Science and Technology (KAUST) O ce of Sponsored Research (OSR) under Award No. URF/1/3454-01-01 3 https://dice-group.github.io/semantic-web-challenge.github.io/