Entity linking is a important for connecting text data and knowledge bases. This poster presents a word, mention and entity joint embedding method, which can be used in computing semantic relatedness in entity linking approaches. Recently, several large-scale Knowledge Bases have been created and successfully applied to many areas, such as DBpedia, YAGO, and Freebase. In many applications of knowledge bases, a basic task is to identify entities in text and linking them to a given knowledge base, which is usually called entity linking. The task of entity linking is challenging because of entity name variations and entity ambiguity. On one hand, one entity can be mentioned in text by different names; for example, both ”Beijing” and ”Peking” can refer to the same entity ”Beijing City”. One the other hand, the same mention can refer to multiple different entities; for example, ”Apple” may refer to ”Apple Inc” and the fruit ”Apple”, etc. Lots of work has been done on the problem of entity linking, [5] gives detailed review of all kinds of entity linking approaches. In the entity linking approaches, computing semantic relatedness between entities and contextual context is very important for entity disambiguation. In this poster, we propose a new way to compute the relatedness. A word, mention and entity joint embedding learning methods is proposed. Based on the results of the joint embedding, different kinds of relatedness among words, mentions, and entities can be easily computed. The rest of this paper introduces the proposed embedding method in detail. In this paper, we propose to use Skip-gram model [3] to jointly map entities, mentions and words to the same low-dimensional vector space. By using the jointly learned vectors, various relatednesses can be efficiently computed, such as entity-word relatedness, mention-word relatedness and entity-entity relatedness.
Introduction 2.1
The Skip-gram model The skip-gram model is a recently published learning framework to learn continuous word vectors from text corpora. Each word in the text corpora is mapped to a continuous embedding space. The model is trained to find word representations that are good at predicting the surrounding words in a sentence or a document. Given a sequence of training words w1; w2; :::; wT , the objective of the model is to maximize the average log probability where c is the size of training context, p(wt+j jwt) is defined as
T O = 1 X
T
X
log p(wt+j jwt) t=1 c j c;j6=0 p(wOjwI ) = exp(vw0O T
vwI ) PwW=1 exp(vw0O TvwI ) (1) (2) where vw and vw0 are the input and output vector representations of w, and W is the number of words in the vocabulary. The learned vectors of words can capture the semantic similarity of words; similar words are mapped to nearby places in a low-dimensional vector space.
r(a; b) =
a b k a kk b k Input
Projection Beijing is the capital of People's Republic of China and …… MEN(Beijing) ENT(Beijing) is the capital of MEN(People'sRepublicofChina) ENT(China) and …… capital
Output MEN(Beijing) ENT(Beijing) is the of MEN(People'sRepublicofChina)
ENT(China) and Skip-gram model is initially designed to learn embeddings of words. In order to extend the word model to a joint model of word, entity and mention, we add mentions and entities in the training corpus which only contains words before. Let the original corpus be C = fw1; w2; :::; wN g, if a certain word sequence s = fwi; :::; wi+kg in C is a mention to an entity e in the knowledge base, we replace s with two tokens fM EN (wiwi+1:::wi+k); EN T (e)g; after that, the original word sequence containing s in C becomes fwi 1; M EN (wiwi+1:::wi+k); EN T (e); wi+k+1g. After annotating all the mentions and their corresponding entities, C is converted to a hybrid corpus C0 that containing words, mentions, and entities. C0 is then used to train the Skip-gram model, which will generate representations in the same vector space for words, mentions, and entities. Figure 1 shows an example of using the Skip-gram model to predict the surrounding tokens of the word capital in the example sentence. 2.3
Using Wikipedia as Training Corpus Annotating mentions and entities in a corpus is a time-consuming task. Fortunately, if we use Wikipedia as a knowledge base, it contains all the annotations we need. Figure 2 shows part of the page of Beijing in Wikipedia and its source text in editing model. In Wikipedia, a internal hyperlink is annotated by [[entity j mention]]; it also could be [[entity]] when the entity is mentioned by the exact name of it. Processing these inner links, we can generate the corpus containing words, mentions and entities together. So in this paper, Wikipedia is used as the target knowledge base that entities link to, and its articles are processed to train the skip-gram model.
'''Beijing''', formerly '''Peking''', is the capital of the [[China|People's Republic of China]]
and one of the most [[List of metropolitan areas by population|populous cities in the world]]
…
To evaluate the effectiveness of the proposed embedding model, we use the embedding
results in a entity linking approach. The entity linking approach was first introduced in
[
In the evaluation, English Wikipedia is used as the target Knowledge Base for entity linking. The dataset of Yahoo Search Query Log To Entities 1 is used for the evaluation. 1 http://webscope.sandbox.yahoo.com/catalog.php
?datatype=l&did=66 This dataset contains manually identified links to entities in Wikipedia. In total, there are 2,635 queries in 980 search sessions, 4,691 mentions are annotated which link to 4,725 entities in Wikipedia.
We also compared our approach with two entity linking systems, Illinois Wikifier [
Table 1 shows the evaluation results of three different approaches. The precision and recall of each approach are evaluated. According to the results, our approach achieves the best precision and recall. It shows that the joint embedding model is effective in entity linking problem.