Can Pattern Learning Enhance Complex Logical Query Answering?

Abstract

Logical patterns, such as symmetry and composition, have been proven to be beneficial in the knowledge graph completion task. However, their influence has been unexplored in first-order logical (FOL) query reasoning methods. In this work, we present an inductive bias for query embedding models, Patternaware Cone Embedding (PConE), to support learning and reasoning with logical patterns. PConE combines the advantages of cones and the rotation operator for powerful algebraic operations for pattern inference. Our experiments demonstrate how the capability to capture logical patterns positively impacts the results of query answering.

Keywords

• Query answering
• Knowledge graphs
• Query embedding

-

CEUR ceur-ws.org

1. Introduction

CEUR Workshop Proceedings © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). q = # $ = ∃&: ()*ℎ,-*-./012/3ℎ& &, ./2,5 673

∨ )*ℎ,-*-./012/3ℎ& &, 972/3- 673 ) ∧ ¬1-=>>=*- 6. ?/0=,5/ , & ∧ @A2*ℎB,=C- ($, &) * $ ( 4 4 ( * /

L. Messi

&'(123-#

Argentina NFT

!"#$ℎ&'()*

!-#.+$ +$ℎ'*$*,-./#-0ℎ1

Rosario

World Cup

/*(4,-./#-0ℎ1 . # 0 1 - /- ℎ +$ℎ'*$*,

J. Mascherano

C. Ronaldo

!-#.+$ Paris !"#$ℎ&'()* +$ℎ'*$*,-./#-0ℎ1 Europe Cup/*(4, 1 ℎ 0 # /*(4,-./ +$ℎ'*$*,-./#-0ℎ1 -./#-0ℎ1

K. Mbappé

&'(123-#

France NFT

!-#.+$

Pepe

Maceió

!"#$ℎ&'()* &'(123-# Portugal NFT &'(123-# /*(44($* !"#$$#%" operators. Examples of patterns found within KGs encompass symmetry (e.g., ), inversion (e.g., and ℎ ℎ ), and composition (e.g., when an athlete a team and ℎ , it implies ℎ ℎ ). As these logical patterns greatly influence the interplay of entities and relations, related work [ 4, 5 ] has demonstrated that an embedding’s ability to support them improves its link prediction quality. However, similar accommodation of logical patterns in the embedding space is still lacking for query embeddings.

In this poster, we propose a novel method, PConE, to support the acquisition and representation of logical patterns for query answering. We describe its basic working mechanism and the evaluation strategy and present experimental evidence to demonstrate its eficiency in handling ifrst-order logical queries with a lightweight structure (with only half the parameters of other baseline models).

2. PConE

PConE defines each relation as a rotation from the source entity set to the answer/intermediate entity set, where entities and entity sets are modeled as vectors and cones, respectively. Each cone q is parameterized by q = (h , h ), where |h |2 = 1, |h |2 = 1 with | ⋅ |2 being the L2 norm, and h , h ∈ ℂ represent the counter-clockwise upper and lower boundaries of the cone, such that h ≡ , h ≡ , where { ,} represent the angle between the boundary and the axis, is the embedding dimension. First-order logical operators, conjunction, disjunction, and #

#

France NFT

1" 1$

Europe Cup

Mbappé

Rosario

Paris

1# (ii) Composition 0&$3."/01"*ℎ2!(x,z) ∧ ;!$2=>"1!(z, y) → (%ℎ!&%&."/01"*ℎ2!(x, y)

K. Mbappé

L. Messi

#$ #" (iii) Inversion D"1/(% (x, y) ⇄ D+1%ℎ;!$#& (y, x) negation, are translated into geometric operators in the complex vector space. We derive the ifnal query embedding by executing geometric operators on the selected entity sets along the computation graph. The model is trained to minimize the distance between the query cone embedding and the answer entity vector. The geometric operators are designed below. Relational Transformation Given a set of entities and a relation, the transformation operator selects the neighboring entities by relation. Existing query embedding methods [ 2, 1, 6, 3 ] apply multi-layer perceptron networks to accomplish this task. They do not accommodate the learning of potential logical patterns which can help in reasoning logical queries. To capture diverse patterns, we represent each relation r ∈ ℂ2× as a counterclockwise relational rotation on query embeddings about the origin of the complex plane such that r = (r , r ), where |r | = 1, |r | = 1, and r , r ∈ ℂ . Given the query embedding q = (h , h ) and a relation r, the transformed query embedding (h′ , h′ ) is defined as h′ = h ∘ r , h′ = h ∘ r . Figure 2 illustrate the relational transformation process in the presence of various patterns. Logical Operators Given the input of multiple entity sets modeled by cone embeddings, the intersection operator computes their intersections through a neural network-based permutationinvariant function. Given the cone embedding of a set of entities q, the negation operator finds its corresponding negation q¬ as the complement of the cone. In addition, given the input of multiple entity sets q1, ..., q , the union operator finds the disjunction set as the union of multiple cone embeddings in the same complex plane.

3. Preliminary Results

We evaluate our model on a wider range of datasets and dataset splits in addition to existing query answering benchmark datasets to thoroughly assess how learning logical patterns afects query answering.

Model Performance Table 1 summarizes the performance of all methods on answering various query types in two benchmark datasets WN18RR [ 7 ] and NELL [ 3 ]. On nearly all query 2p 4.6 9.8 12.3 14.5 17.6 14.5 13.0 16.1 17.7 types, PConE consistently outperforms all baseline approaches.

Respective Influences of Logical Patterns Tono bqeutetreiresstiundvyoltvhiengsploecgiificcaimlppaatctetronfsP,CaomnoEre 4308 PBCasoenlEine lisdanwat-tiedaoresniepnsttgihinndvtaaoontla,fivavseleeydctsaNiitsne,EagitLsnhoLdrem.ieqℎWas u,deeer ci eao,stfe.bgtaThosheer,eid zqseouuntebhrtgehyreotaeurnsept--, rcccyauA%33332460 34.43%1.4% 28.6%28.2% 34.83%2.2%28.92%9.7% volveiasllathcorenejulongcitcivaelpsaettteorfnqs.uTerhieescathteagtoirny- 2286 26.22%5.5% ℎ corresponds to queries that do not in- Inverse SQyumemrieestrwyiCtho mreplaotsiiotnionpatteJroninst Others volve any of these logical patterns. Figure Figure 3: Average performances of PConE and 3 shows the average performances of PConE Baseline model (ConE) over query suband neural baseline model on these subgroups. groups with diferent logical patterns. It is observed that PConE outperforms the neural baseline model on queries that had logical patterns, especially inverse relations. However, PConE does not generalize as well to queries that were not influenced by logical patterns compared to the baseline model.

4. Conclusion

Logical patterns in complex query answering remains understudied. To the best of our knowledge, PConE is the initial study to investigate how logical patterns improve logical query reasoning. On the other hand, due to natural geometry features, the relational rotational projection can only be used to cone embedding. We will develop more generic and efective ways to improve relation pattern learning in complex query reasoning.

5. Acknowledgement

The authors thank the International Max Planck Research School for Intelligent Systems (IMPRSIS) for supporting Yunjie He, Bo Xiong and Yuqicheng Zhu.

References

[1] H. Ren , W. Hu , J. Leskovec, Query2box: Reasoning over knowledge graphs in vector space using box embeddings, in: ICLR, OpenReview .net, 2020 .

[2] Z. Zhang , J. Wang , J. Chen , S. Ji , F. Wu , Cone: Cone embeddings for multi-hop reasoning over knowledge graphs , in: NeurIPS , 2021 , pp. 19172 - 19183 .

[3] H. Ren , J. Leskovec , Beta embeddings for multi-hop logical reasoning in knowledge graphs , in: NeurIPS , 2020 .

[4] Z. Sun , Z. Deng , J. Nie , J. Tang , Rotate: Knowledge graph embedding by relational rotation in complex space, in: ICLR (Poster), OpenReview .net, 2019 .

[5] M. Nayyeri , C. Xu , Y. Yaghoobzadeh , S. Vahdati , M. M. Alam , H. S. Yazdi , J. Lehmann , Lossaware pattern inference: A correction on the wrongly claimed limitations of embedding models , in: PAKDD (3) , volume 12714 of Lecture Notes in Computer Science, Springer, 2021 , pp. 77 - 89 .

[6] W. L. Hamilton , P. Bajaj , M. Zitnik , D. Jurafsky , J. Leskovec , Embedding logical queries on knowledge graphs , in: NeurIPS , 2018 , pp. 2030 - 2041 .

[7] Z. Huang , M. Chiang , W. Lee , Line: Logical query reasoning over hierarchical knowledge graphs , in: KDD, ACM , 2022 , pp. 615 - 625 .