Knowledge Graphs (KGs) play a key role in many arti cial intelligence applications. Large KGs are often constructed through a noisy automatic knowledge extraction process. Noise detection is, therefore, an important task for having high-quality KGs. We argue that the current noise detection approaches only focus on a speci c type of noise (i.e., fact checking) whereas knowledge extraction methods result in more than one type of noise. To this end, we propose a classi cation of noise found in automatically-constructed KGs, and an approach for noise detection focused on speci c types of noise.
Knowledge Graphs (KGs) are key components of most modern arti cial intelligence and cognitive applications. These KGs are largely constructed from textual corpora using automatic information extraction techniques. Such techniques introduce noise in the KGs and noise detection and removal become essential steps.
Most of the current noise detection in knowledge graphs is done with human supervision. For instance, large KGs such as YAGO2, DBpedia, or Wikidata use human contributors to verify the correctness of a given fact. This is a timeconsuming task and requires a lot of human e ort for large KGs. Although crowd-sourcing could be a solution for public general-domain KGs, it may not be a viable solution for enterprise KGs, due to both privacy issues as well as the need to rapidly create KGs from a large number of distinct corpora and in speci c domains that require deep expertise. Thus, there is a need for automatic techniques for detecting noise in KGs.
Current automatic noise detection techniques are focused on factual correctness of triples. In this paper, we discuss the need for di erent types of noise in KGs and how to detect those speci c types of noise. ? This research is partially supported by the 4V project (TIN2013-46238-C4-2-R) and the BES-2014-068449 grant. Work done while the author was an intern at IBM.
Mihindukulasooriya et al.
Knowledge Graph Re nement [
Based on the analysis of the output of a commercial information extraction framework, which parses text corpora and produces triples, we de ned the categories in Fig. 1. Out of these, we identify Factual True triples as the most relevant and Inconsistent, Generic, Factual False categories as noise.
Inconsistent triples contradict the domain model they represent. As such, these triples are completely implausible and meaningless. For instance, (Barack Obama, siblingOf, White House), is not plausible. If siblingOf relation is speci ed formally to have a range of Person and if Person and Building are disjoint, that leads to a logical inconsistency. Nevertheless, those granular axioms are not Towards Comprehensive Noise Detection in Automatically-Created KGs always present for such reasoning and data pro ling can be used to identify common patterns in data that can provide heuristics of inconsistencies.
Generic triples do not mention speci c entities and have less information value. For example, (family, residesIn, New York ). Nevertheless, if both entities are generic such triples can provide schema-level information, for example, (family, residesIn, city ). Finally, factually false triples are the ones that contain incorrect information. For example, (Boston, capitalOf, USA) is factually false. 4
We propose to lter inconsistent and generic triples prior to fact checking as illustrated in the Fig. 2. This section describes approaches for detecting inconsistent and generic triples using the knowledge in external KGs.
Inconsistent triple checking is performed by mapping both entities as well as relations to external KGs and considering both ontological axioms that de ne formal conceptulizations (e.g., domain and range of properties or disjoint types) and common patterns in data. A relation mapping is one-to-one if both relations in information extraction and properties in KG have same granularity, for example, siblingOf to dbo:sibling. Otherwise, the mapping is conditional based on the domain class, for example, partOfMany maps to dbo:country in dbo:City class and dbo:album in dbo:Single class.
Generic triple detection is performed using part-of-speech tagging using NLP tools. We use the intuition that when the subject or the object is not a proper noun, it typically refers to a generic entity rather than a speci c one. If either the subject or the object is generic, we label the triple generic. For determining factual correctness, we use a similar approach to fact checking by looking for evidences that con rm a given triple in an external KG using entity disambiguation and relation mapping described in Algorithm 1.
Fact checking is performed by nding evidences con rming a given triple in external knowledge graphs similar to the approaches in Section 2. We conducted preliminary experiments to evaluate the algorithms using 2,342 manuallylabelled triples. Each triple was labelled with its type (i.e., Inconsistent, Generic, or Factual ) and its truth value (i.e., True, False, or True in the past ) by human annotators. The results are presented in Table 1.
Mihindukulasooriya et al.
Algorithm 1: Inconsistent triple detection Category Inconsistent triple detection Generic triple detection
Con guration Standford NLP Open NLP Combined
Precision 86.84% 85.65% 78.16% 98.25%
Our intuition is that the more-speci c noise detection will improve the overall quality of KGs. One challenge for evaluating this hypothesis is the lack of a large gold standard with noise types. We plan to create one using crowd-sourcing.
Further, we also plan to test another alternative approach based on graph embeddings to demote triples that are inconsistent with other triples. For this, we learn representations for entities and relations by constructing functions that represent interactions between related entities.