Entity-centric queries constitute a signi cant proportion of all search queries processed by the popular search engines. Answering such queries often involves selecting facts pertaining to an entity from an underlying knowledge graph. Prior work on this draws on hand-crafted features that require scanning the entire knowledge graph beforehand. Instead, we propose a neural method that exploits the linguistic and semi-linguistic nature of the entity search queries and the facts, and can hence be applied dynamically to entirely new sets of candidate facts. We optimize our model using a pairwise loss function to correctly predict the relevance and importance scores for each fact for a given query entity, while the overall fact ranking is based on a linear combination of these scores. We show that our simple approach outperforms previous work, ensuring better fact retrieval for entity-centric search queries.
In recent years, knowledge cards have become an integral part of popular Web
search engines. Knowledge cards are information boxes that appear on the search
engine result pages when a user searches for entity-related information [
In practice, di erent entity-related queries may pertain to quite di erent
aspects of an entity. A search engine query such as \einstein education" ought
to give preference to other facts than a query such as \einstein family ". To
address this task of dynamic query-speci c fact ranking, Hasibi et al. proposed
a model called DynES [
However, DyNES is based on hand-crafted features that are cumbersome to compute, as they need to be extracted beforehand from the set of all facts in the large-scale knowledge graph, rendering this method unsuitable for ad hoc 1 Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). settings. Moreover, DynES performs a simple pointwise ranking of the facts, where each fact is considered in isolation, using Gradient Boosted Regression Trees, which learn an ensemble of weak prediction models.
In contrast, we propose a novel model that obviates the need for a
cumbersome process of extracting hand-crafted features from the large knowledge graph.
Our key contributions are as follows. (1) We propose a deep neural model with
a pairwise loss function to address the task of query-dependent fact retrieval for
entity-centric search queries. (2) Rather than depending on a large knowledge
graph for feature extraction, our model draws on recent advances in
Transformers with self attention [
In the following, we de ne relevant terminology that is used in the remainder of the paper. We consider a fact f as a predicate{object pair returned when a query is made with regard to an entity, with that entity serving as the subject. De nition 1. (Importance) Importance is an attribute of a fact f that determines its relation to the subject entity s in absolute terms, irrespective of the provided query. It is denoted as is(f ).
De nition 2. (Relevance) Relevance, in turn, describes to what extent a given candidate fact f is pertinent with regard to a given natural language search query q issued by the user along with the entity s as the subject. It is denoted as rs;q(f ). De nition 3. (Utility) The overall utility of a fact f with respect to a query q and entity s is de ned as a weighted sum of the importance and relevance scores of the fact with respect to query and entity. It is denoted as us;q(f ) and computed as us;q(f ) = is(f ) + rs;q(f )
The weights , may be adjusted freely to account for application scenariospeci c considerations. Thus, utility relates the fact to the query in a more comprehensive manner than the importance and relevance scores alone can. 3
Given the natural language input query Q as well as a candidate fact fi =
hp; oi 2 F , where F is the set of all candidate facts for Q, our model accepts the
query along with the natural language labels of p and o and invokes BERT [
Before feeding the tokenized sequences Q, P , O to the model, special tokens [CLS] and [SEP] are inserted into the input sequence. The [CLS] token signi es the start of each sequence, while the [SEP] token serves as a demarcation point separating the query segment from the fact segment in the input sequence. The resulting sequence of input token identi ers now becomes: [CLS], q1; : : : ; ql, [SEP], p1; : : : ; pm, o1; : : : ; on, [SEP]
The most relevant component for our task lies in the encoded representation of the [CLS] token, which serves as a representation of the entire input sequence. This representation is passed through a fully-connected layer followed by a sigmoid activation function to yield a ranking score, which is then compared to the ground truth. Formally, g(fi) = (Whp + b), where hp 2 Rd is the [CLS] representation from the nal hidden layer of the BERT encoder, W 2 R1 d and b are trainable parameters, and (x) = 1+1e x .
The model is trained to minimize a pairwise ranking loss that considers pairs of facts (fs and fi) and encourages the model to predict scores for the two involved facts that re ect the correct relative ordering between them. Ideally, the di erence between the two predicted scores (g(fs) g(fi)) should equal the di erence (r(s) r(i)) between the corresponding ground truth ranking scores. These di erences are computed as signed values rather than absolute values, so the ordering is crucial. Based on this intuition, we de ne the loss function as a pairwise mean squared error as follows:
h r(s) r(i) g(fs) g(fi) i2 L(g; F ; R) = Psn=11 Prn(i)i=<1r(s)
The nal ranking is created by ordering the candidate facts fi 2 F by g(fi) in descending order, breaking ties arbitrarily. Thus, if g(fi) > g(fj ), then fi should be ranked higher than fj . 4
We perform our experiments on two dataset variants put forth by Hasibi et al. [
We compare several di erent models, including DynES [
Model
RELIN [
The results of our experiments on the Complete Dataset and URI-only Dataset are given in Tables 1 and 2, respectively. Our model outperforms the DynES model with absolute gains of 4:9% and 13:2% in terms of the NDCG@10 metric on the Complete Dataset for the utility and importance-based rankings (as de ned in Section 2), respectively. We observe a similar trend for 2 https://github.com/AtharvaParanjpe/Dynamic-Fact-Ranking-For-Entity-Centric-Queries the URI-only Dataset, where our model consistently outperforms the DynES model, with respective absolute gains of 4:3% and 2:0% in the NDCG@5 metric for utility and importance rankings.
The moderate performance of the pre-trained BERTBASE model suggests that BERTBASE already has su cient linguistic information embedded in it to be able to rank the facts to a certain degree. In fact, without any ne-tuning, BERTBASE outperforms the BiLSTM Dual Encoder baseline in most of the cases.
Our model outperforms the pointwise ranking variant with as high as 8:5% and 5:7% absolute gain in NDCG@5 and NDCG@10 metrics for the utility scores on the URI-only Dataset. We conjecture that this is because a pairwise loss function allows the model to better assess the di erences between di erent facts and because this training regime better exploits the available training data. 5
In this paper, we propose a new neural method to learn entity-centric fact rankings, accounting for both the saliency and the relevance of facts with regard to the query. Our method adopts a pairwise ranking approach while drawing on state-of-the-art deep neural modeling techniques to analyze the semantics of queries and candidate facts along with their semantic connections. Unlike previous work, it can dynamically be applied to entirely new candidate facts without the need to compile knowledge graph statistics. In our experimental evaluation, we observe substantial improvements over previous work.