=Paper=
{{Paper
|id=Vol-2456/paper15
|storemode=property
|title=Learning Ontology Axioms over Knowledge Graphs via Representation Learning
|pdfUrl=https://ceur-ws.org/Vol-2456/paper15.pdf
|volume=Vol-2456
|authors=Leyuan Zhao,Xiaowang Zhang,Kewen Wang,Zhiyong Feng,Zhe Wang
|dblpUrl=https://dblp.org/rec/conf/semweb/ZhaoZWFW19
}}
==Learning Ontology Axioms over Knowledge Graphs via Representation Learning==
https://ceur-ws.org/Vol-2456/paper15.pdf
Learning Ontology Axioms over Knowledge
Graphs via Representation Learning
Leyuan Zhao1,3 , Xiaowang Zhang1,3,? , Kewen Wang2 , Zhiyong Feng1,3 , and
Zhe Wang2
1
College of Intelligence and Computing, Tianjin University, Tianjin 300350, China
2
School of Information and Communication Technology, Griffith University,
Brisbane, QLD 4111, Australia
3
Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin, China
?
Corresponding author: xiaowangzhang@tju.edu.cn
Abstract. This presents a representation learning model called SetE by
modeling a predicate into a subspace in a semantic space where entities
are vectors. Within SetE, a type as unary predicate is encoded as a set
of vectors and a relation as binary predicate is encoded as a set of pairs
of vectors. A new approach is proposed to compute the subsumption of
predicates in a semantic space by employing linear programming methods
to determine whether entities of a type belong to a sup-type and thus an
algorithm for learning OWL axioms is developed. Experiments on real
datasets show that SetE can efficiently learn various forms of axioms
with high quality.
1 Introduction
Ontology construction is a core task of ontology engineering. It has been a re-
search challenge in both knowledge representation and machine learning com-
munities. This is because ontologies are often based on logical formalisms such as
description logics (DLs), and contain more complex logical structures than graph
databases or RDF triples. DL-Learner [1] is among the first practical systems
to learn ontological expressions, including complex DL class descriptions. Many
methods for learning new first order formulas and rules have been developed in
Inductive Logic Programming (ILP) but they are often unable to handle very
large ontologies. Recently, some attempts have been made to effectively learn
rules, such as [4], over KG through techniques in knowledge representation learn-
ing, but the rules they learn are not typical ontological axioms. What is more,
conventional embedding models (e.g.,TransE, TransR, DistMult and SimplE)
mainly focus on KG completion, which only embed entities and relations without
modeling unary predictes. TransC [3] firstly differentiate types (unary predictes)
and entities, it encodes each type as a sphere and can learn the SubClassOf rela-
tionship between types. However, the encoding of relations and types in TransC
*
Copyright 2019 for this paper by its authors. Use permitted under Creative Com-
mons License Attribution 4.0 International (CC BY 4.0).
2 L. Zhao et al.
is split, which prevents it from learning the relation SubPropertyOf and other
complex axioms (e.g.SubClassOf(ObjectSomeValuesFrom(P, C), D)).
In this paper, we propose a novel unified embedding (called SetE) for KG
unary predicates (types) and binary predicates (relations) treating types as sets
of entities and relations as sets of entity pairs. On this basis, the subsumption
is transformed to relative position of set boundaries which can be efficiently
computed by linear programming (LP). We provide an algorithm for learning
positive OWL axioms over large-scale knowledge graphs.
2 Our Approach
In this section, we will introduce SetE and the learning algorithm.
Baguette .. 0.2 …. t
f(e,t)
p
g(s,p,o)
Quartz
.. 0.8 …. e s o
(a) A simple illustration of the embeddings (b) The structure of SetE
of Baguette and Quartz.
Fig. 1. An overview of SetE.
Embedding model Inspired by [2], we treat each dimension of embedding
as a feature describing certain unique aspect of characteristics. For example, as
shown in Figure 1(a), assume the third dimension representing the feature of
hardness, then the value of entity Quartz in this dimension is greater than the
value of entity Baguette.
We use inner products to capture feature interactions between sets and their
elements(in this case, types and entities). Because when they have common
features, the inner product gets a larger value, as illustrated in Figure 1(b). An
Thus the score function of the instance fact < e, InstanceOf, t > is defined as
follows, where e and t are embedding of e and t, resp.; n is the dimension of e,
n
X
T
f (e, t) = e t = [e]i ∗ [t]i . (1)
i=1
Following the same intuition, the entity pair < s, o > can be considered as an
instance of the relation p, so we model the fact < s, p, o > as follows. Where s
and o are head and tail entity of the relation p, concate(s, o) means concatenate
the two vecotrs s and o.
2n
X
g(s, p, o) = concate(s, o)T p = [concate(s, o)]i ∗ [p]i (2)
i=1
Learning Ontology Axioms over Knowledge Graphs via Embdding 3
To train the model, we introduce type boundary Bt ∈ R. So that for all entity e
of type t, there has f (e, t) > Bt ; for e ∈
/ t, there has f (e, t) < Bt . The relation
boundary Br is the same. Like previous models, we generate negative samples
and use SGD to train SetE.
LP to Subsumption Subsumption in KG has SubClassOf and SubPropertyOf.
We take SubClassOf as an example to show how this can be transformed into
LP under our model. The axiom SubClassOf(C, D) means that all entities that
are instances of C must be instances of D. i.e., f (e, tC ) > Bt implies f (e, tD ) >
Bt , where tC and tD are type embeddings of C and D. So we convert this to
linear programming that computes the minimum value of f (e, tD ) subject to
e ∈ C (f (e, tC ) > Bt ). If the minimum value is greater than the boundary
Bt , that is for all entity e in type C, e always satisfy D, so we get the axiom
SubClassOf(C, D).
Learning Ontology Axioms Based on previous analysis, we use liner pro-
gramming on embeddings to learn the following forms: A1 , SubClassOf(C, D);
A2 , SubPropertyOf(P, Q); A3 ,SubClassOf( ObjectSomeValuesFrom(P, C), D);
A4 , SubClassOf(ObjectIntersectionOf( C, D), Range(F )). The algorithm learning
Sub ClassOf(C, D) is as follows.Line 3 means that if values in tC are smaller
than or equal to tD in every dimension, then we can directly get that for any
e, if f (e, tC ) > Bt then f (e, tD ) > Bt . At last, Filter() returns axioms whose
SC(standard confidence, defined in [4]) are greater than M inSC.
Algorithm 1 Learning SubClassOf Axioms from a KG
Input: a KG K, and two real numbers LBt and M inSC ∈ [0, 1]
Output: a set O of SubClassOf axioms
1: E := SetE(K); O := ∅.
P embeddings tC and tD in E do
2: for type
3: if ( ni=1 ([tC ]i 6 [tD ]i )?1 : 0) == n then
4: Add SubClassOf(C, D) to O
5: else if LP(tC , tD ) > LBt then
6: Add SubClassOf(C, D) to O
7: end if
8: end for
9: O := Filter(O, M inSC)
10: return O
3 Experiments and Evaluation
The experiment on YAGO39K aims to evaluate the effectiveness of SetE by
comparing with the state-of-the-art model TransC[3] in SubClassOf classifica-
tion. We retain four metrics: Accuracy, Precision, Recall and F1-score. TransC
was trained with the configuration in their report. SubClassOfs were removed
4 L. Zhao et al.
from the training set. To reflect real data that the negative samples far exceeds
the positive(e.g.,#negative :#positive is 226:1 in DBpedia 2016 OWL), we add
the proportion of negative samples during the experiment.
Result in Table 1 indicates that SetE outperforms TransC and is getting
better when improving the proportion of negative samples. The Precision of
SetE is much higher than TransC (up to 87.03% for rate 1:10). It shows that
SetE is more cautious in making positive judgments, i.e., SetE distinguishes
positive samples better.
Table 1. Classification results(%) based on the rate of #positive and #negative.
Model rate Accuracy Precision Recall F1
1:1 57.95 56.61 68.10 61.82
TransC 1:4 50.96 24.20 68.10 35.71
1:10 48.17 11.23 68.10 19.28
1:1 65.20 70.10 53.10 60.41
SetE 1:4 80.78 51.91 53.10 52.50
1:10 87.03 35.66 53.10 42.67
4 Conclusion
In this paper, we present a new model SetE to specifically represent types and
relations in a semantic space which can reduce subsumption into linear program-
ming. Our proposal utilizes the logical relationship to characterize the semantic
features of expressive types in learning shows certain interpretability. In the fu-
ture, we will improve the quality of expressive axioms learned and considerate
even negated axioms.
Acknowledgments
This work is supported by the National Key Research and Development Program
of China (2017YFC0908401) and the National Natural Science Foundation of
China (61976153,61972455). Xiaowang Zhang is supported by the Peiyang Young
Scholars in Tianjin University (2019XRX-0032).
References
1. Bühmann, L., Lehmann, J., and Westphal, P.: DL-Learner - A framework for in-
ductive learning on the semantic web. J. Web Semant., (39),15–24 (2016).
2. Ding, B., Wang, Q., Wang, B., Guo, L.: Improving knowledge graph embedding
using simple constraints. In: Proc. of ACL 2018, pp.110–121.
3. Lv, X., Hou, L., Li, J., Liu, Z.: Differentiating concepts and instances for knowledge
graph embedding. In: Proc. of EMNLP 2018, pp.1971–1979.
4. Omran, P.G., Wang, K., Wang, Z.: Scalable rule learning via learning representation.
In: Proc. of IJCAI 2018, pp.2149–2155.