The basement of knowledge base question answering (KBQA) is to understand the given question and extract the meaning from it. Existing works largely focus on generating query graphs to represent the semantics of the question while ignoring understand the real meaning of it. To augment question understanding, in this paper, we leverage rich external linguistic knowledge to enhance question semantics. First, we integrate the sememe and gloss information into words, where sememe (the minimum semantic units of word meanings) and gloss (sense de nition) are used to disambiguate the word sense and enrich questions information. Moreover, we present a co-attention network to build co-dependent representations for the sememe and gloss. Experiments evaluated on two data sets show that our model outperforms existing approaches.
Semantic parsing is an important approach to KBQA, which constructs a query structure (called query graph) that represents the semantics of questions. Semantic parsing based approaches e ectively transform questions into logical forms where the reliability of logical forms can ensure the correctness of answering questions. The success of semantics parsing lies in representing the semantics of questions to better capture users' intention.
However, in recent years, many semantic parsing approaches focus on
complex query graph generation and re-ranking[
In this paper, we propose an augmented question representation method by
leveraging the sememe and gloss information. Speci cally, we take advantage
of the gloss information from WordNet and the sememe [
RGCN layer
...
Question：where was president
Co-Attention
Ug 1.the person who holds the office of head of state of the
United States government 2.an executive officer of a firm
or corporation 3.the head administrative officer of a college or
...
university Gloss selector cos similarity + avg pooling Relation2...
Relation1 Us from? context
Head of government head of government president (state president) president (principal) ...
president (CEO)
Senses
Layer peh/oumlmiatiancnas/g/oHec/eccauodpuOanttfirSoytn.a./t. /heudmsutcuaandty/ioo/tcnec/aoucfphfai.c.t.iioanl/ ... /hmeucamonnaaognme/o/pycrc/oiumfpfaaicrtiyiao..ln/. SeLmaeymeres
Sense selector
Given a question Q = fw1; : : : ; wng, we generate its candidate query graph set
by method in [
Sense Selector: This selector selects each word sense that is most relevant to the context. We rst feed Q into a bi-directional long short-term memory network (Bi-LSTM) to generate the hidden representation fh1; : : : ; hng. Then we generate the context representation of Q as below:
n n context = X softmax(tanh(hi> ( 1 X hi))) hi (1) i=1 n i=1
To calculate the correlation between each sememe and context, we use Sigmoid function to obtain probability value by: p(eij jcontext) = Sigmoid context ei>j ; 8 j 2 (1; : : : ; k) (2) For each sense in Swi , its probability is calculated by the average of all the sememe probabilities that it contains. In this way, we can select the sense with the highest probability value under the current context and denote it as Smwiax = fe1; : : : ; ekg, where fe1; : : : ; ekg represents all sememe vectors it contains. Gloss Selector: This selector selects the gloss that is most relevant to the selected word sense. Analogously, we use Sigmoid function to obtain the highest probability of gloss that is most relevant to the average embedding of Smwiax. We denote the selected gloss as Gwmiax = fo1; : : : ; omg, where fo1; : : : ; omg represents all word embeddings it contains.
Co-Attention: To model the mutual in uence and highlight the important information in the sememe and gloss, we introduce a co-attention network to dynamically combine the sememe and gloss representation as: (3) (4) (5) U s = tanh(Smwiax
Gwmiax>);
U g = tanh(Gwmiax Smwiax>) SGwi = k X[U s softmax(U:s)]:j + (1 j=1
m ) X[U g softmax(U:g)]:j j=1 where SGwi is the combination representation of the selected sememe and gloss of word wi, is the parameter and 2 [0; 1]. softmax(U:s) and softmax(U:g) are attention weight matrix for softmax function across each column of U s and U g, respectively. []:j denotes j-th column of []. Finally, we concatenate SGwi to the word embedding of wi to enrich the semantics and reduce ambiguity. Question Representation: We treat fx1; : : : ; xng as the initial representations of the given question which has been integrated with sememe and gloss. To further augment the contextual embeddings of the question, we parse the question into its syntactic dependencies graph DG and adopt relational graph convolutional network (RGCN) to digest this structural information: xi(l+1) = ReLU @rX2R jX2Nir jN1irj Wr(l)x(jl) + W0(l)xi(l)A 0 1 Here R is a set of dependency-relation. l is l-th layer, Nir is the set of all rneighbors of i-th node in DG. Note that W0 and Wr are weighted matrixes. Finally, we apply a pooling operation after the last RGCN layer to get the representation of the question.
Relation Representation We represent relations from di erent granularity in a query graph. For each relation, we take its relation-level and word-level representations into consideration. The word-level relation is calculated by its average word embeddings. The relation-level representation is the vector of unique token of relation name. Then, each relation is represented by the sum operation and we perform max pooling over relations to obtain the nal relation representation. 3
We use Wikidata as our KB and conduct on two data sets, namely,
WebQSPWD (WSPWD)[
By Table 1, we show that our model outperforms all datasets. Our model achieves 54:2%, 23:3%, 8:9%, 11:9% higher F1-score compared to STAGG, HRBiLSTM, GGNN, Slot-Matching on WSPWD. Analogously, we achieve 59:3%, 45:7%, 39:1%,21:7% higher F1-score on QALD-7. We observe that if we only integrate sememe information \+sememe" or gloss information \+gloss", our model performs worse but still keep competitive. We can conclude the e ectiveness of our augmented question representation with sememe and gloss integration.
Model WSPWD QALD-7
STAGG(2015) [
GGNN(2018)[
To measure the performance across questions of di erent complexity, we break down the performance by the number of relations that are needed to nd the correct answer on WSPWD, and results are shown in Fig. 2, we can see that our model is e ective in dealing with di erent complexity questions. 4
In this paper, we augment question understanding to improve KBQA. The sememe and gloss information can bene t from each other and enhance question semantics. In this way, our approach provides a new method of usage of external knowledge in question representation. In future work, we are interested in extending our model for more complex practical questions. 5
This work is supported by the National Key Research and Development Program of China (2017YFC0908401) and the National Natural Science Foundation of China (61972455). Xiaowang Zhang is supported by the Peiyang Young Scholars in Tianjin University (2019XRX-0032).