In this poster, we present a sememe-based approach to semantic parsing in question answering over knowledge base by leveraging a sememe-level semantics to improve the performance of semantic similarity between question and relations. Firstly, we propose a double-channel model to extract both sememe-level semantics and word-level semantics. Moreover, we present a context-based representation to encode the sememe of questions to re ne sememe incorporation for reducing noise. Finally, we introduce a hierarchical representation to encode the sememe representation of relations to remove the ambiguity of words maximally. Experiments evaluated on benchmarks show that our model outperforms o -the-shelf models
Knowledge base question answering (KBQA) is the task of accurately and concisely answering a natural language question over knowledge base (KB) by understanding the intention of the question. As a critical branch of KBQA, semantic parsing based approaches construct semantic parsing trees or equivalent query structures (also called query graph) to represent the given question, and then ranking them by calculating the semantic similarity with the question.
Most current works focus on selecting the semantic relations that most similar
to a question to nd the optimal query graph. Unfortunately, those existing
approaches are limited in di erentiating two relations with the similar word-level
semantics due to the following issues: (
To overtake the above limitations, we leverage sememe from external lexicalsemantic resources. Sememes are minimum semantic units of word meanings [3], a word may have multiple senses, and a sense consists of several sememes. In this poster, we present a sememe-based approach to semantic parsing in KBQA by leveraging a sememe-level semantics. ?? Copyright 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). softmax
Column-wise Max-pooling ...
Sememe
Cosine similarity word-channel
question representation
tokens sememe-channel question representation Tanh(STUQ) ...
LSTM ...
Question
LSTM score +
Sense layer avg sememe-channel + relation representaavgtion WHOORMLED HOME WORLD relation
level
Cq Sememe layer word-channel
relation representation +
avg
Word level
Our model is shown in Fig. 1. In this paper, based on a heuristic algorithm
in [2], we generate candidate query graphs with considering ve kinds of
semantic constraints: entity, type, temporal (explicit and inexplicit time), order, and
compare.
In this part, we generate word-channel representation of question and
relations. Given a question fw1; w2; :::; wng, we feed it into a bi-directional long
short-term memory network (Bi-LSTM) to generate the hidden representation
Q = (h1; : : : ; hn) and obtain hq after pooling operation. Then we transform
hq with a fully connected layer and a ReLU function to get the word-channel
representation of the question:
qw = ReLU (Wq hq + b1)
(
To encode the word-channel relation representation, we take the relationlevel (e.g. \contained by ") and word-level(e.g. \contained ",\by ") relation names into consideration. Given relations fr1; r2; :::; rng in a query graph, for relationlevel representations, we simply take each relation name as a whole unit, and translate it into vector representation as fr1rl; r2rl; :::; rnrlg. For word-level representations, we represent the word sequence of each relation using word averaging as fr1wl; r2wl; :::; rnwlg. We get the nal vector of each word-channel relation representation as ri = rirl + riwl. Finally, we apply pooling operation over all relations and obtain the word-channel representation of relations, denoted by rw. 2.2
Sememe-Channel Representation Sememe-Channel Question Representation We denote Sm as the set of all sememes occurring in the question. Then we map Sm into the vectors S = (s1; : : : ; sn) and adopt a context attention mechanism to deemphasize irrelevant sememes and focusing on more correlative to context ones. The interactive context matrix is calculated as Sq = tanh(S>U Q) . Then we obtain the vector sq by the column-wise max-pooling operation over Sq and use the softmax function. Finally, we get the sememe-channel question representation as following: qs = Wsq(softmax(sq) S) + b2 where Wsq is the parameter matrix.
Sememe-Channel Relation Representation In this part, we adopt a hierarchical attention method to obtain the sememe-level representation of relations sense as a set of sense vecand maximally remove the ambiguity. We denote Rwij tors of word wij as Rwij sememe as a set of sense := fseij1; : : : ; seijkg. We denote Rsijk
sememe := fsmijk1; : : : ; smijkmg. sememe vectors of sense sijk as Rsijk
To obtain the context information of the given question, we denote its word embeddings average as qavg and construct a context representation Cq as following:
Cq = n X softmax(tanh(wi> qavg)) wi i=1 sense is represented as below: Through this, the vector of sense seijk in Rwij
m seijk = X softmax(Wsm tanh(smi>jkc Cq) + b3) smijkc
c=1 where Wsm is a weight matrix. And the representation of the j-th word in the i-th relation is a weighted sum of its senses fseij1; : : : ; seijkg: rij = k X softmax(Wse tanh(sei>jy Cq) + b4) seijy y=1
Finally, we can apply average operation over all words in all relations and obtain the sememe-channel representation of relations, denoted by rs. In this way, we can compute the semantic similarity score of two channel as following: Score = cos (qw; rw) + cos (qs; rs) : (6) 3
Due to Freebase no longer up-to-date, including the unavailability of APIs and
new dumps, we use the full Wikidata dump as our KB. We conduct our
experiments, namely, WebQuestionSP (WebQSP),QALD-7 (Task 4, English). We use
sememe annotations in HowNet for sememe-channel representation.
(
In this poster, we present a sememe-based approach to di erentiate relations with similar semantics in KBQA, where sememe can be leveraged as the minimal semantics of words as an extra natural knowledge to enrich semantics for parsing. In future work, we are interested in maximizing the sememe-level semantics in overtaking the weakness of the word-level semantics in KBQA.
This work is supported by the National Key Research and Development Program of China (2017YFC0908401) and the National Natural Science Foundation of China (61972455). Xiaowang Zhang is supported by the Peiyang Young Scholars in Tianjin University (2019XRX-0032).