Entity relatedness has emerged as an important feature in a plethora of applications such as information retrieval, entity recommendation and entity linking. Given an entity, for instance a person or an organization, entity relatedness measures can be exploited for generating a list of highly-related entities. However, the relation of an entity to some other entity depends on several factors, with time and context being two of the most important ones (where, in our case, context is determined by a particular corpus). For example, the entities related to the International Monetary Fund are di erent now compared to some years ago, while these entities also may highly di er in the context of a USA news portal compared to a Greek news portal. In this paper, we propose a simple but exible model for entity relatedness which considers time and entity aware word embeddings by exploiting the underlying corpus. The proposed model does not require external knowledge and is language independent, which makes it widely useful in a variety of applications.
Entity relatedness is the task of determining the degree of relatedness between
two entities. Measures for entity relatedness facilitate a wide variety of
applications such as information retrieval, entity recommendation and entity linking.
Traditional approaches consider the structural similarity in a given graph and
lexical features [5, 6, 10{12, 15], or make use of Wikipedia-based entity
distributions and embeddings [
Recent works on related topics have supported the hypothesis that the
context of entities is temporal in nature. Speci cally, [
On the other hand, entity relatedness is also strongly dependent on the corpus context at hand. For instance, given a search application operating over a collection of German news articles from summer 2014 and the query entity 2014 FIFA World Cup (https://en.wikipedia.org/wiki/2014_FIFA_World_Cup), a list of related entities should include Germany national football team, Argentina national football team, Mario Gotze, and Brazil. However, considering a collection of Greek articles of the same time period, the top related entities might include Greece national football team, Costa Rica national football team, and Sokratis Papastathopoulos (entities that are not important or popular in German news).
To this end, our work considers entity relatedness as a measure which is
strongly dependent on both the time aspect and the corpus of the underlying
application. Contrary to existing approaches that train embeddings on
generalpurpose corpora, in particular Wikipedia [
The remainder of this paper is organized as follows: Section 2 de nes the problem of time-aware and corpus speci c entity relatedness, Section 3 details the proposed method, Section 4 presents evaluation results, and nally Section 5 concludes the paper and discusses interesting directions for future research. 2
Let D be a corpus of documents, e.g., a collection of news articles, covering the
time period TD = [ s; e] (where s; e are two di erent time points with s < e).
For a document d 2 D, let Ed denote all entities mentioned in d extracted using
an entity linking method [
Consider now a collection of documents D covering a time period TD and
a set of entities ED extracted from and prevalent in D. Given i) one or more
query entities Eq and ii) a time period of interest Tq TD, the task
\timeaware and corpus-speci c entity relatedness" focuses on nding a top-k list of
entities Ek ED related to the query entities Eq in terms of both Tq and D. We
model this task as a ranking problem, where we rst generate a list of candidate
entities Ec ED and then rank this list of entities based on their relevance to
the query entities. For generating the list of candidate entities, one can follow
an approach similar to [
Word embeddings provide a distributed representation of words in the semantic
space. We generate word vectors following the distributional models proposed
in [
We group all documents in D into time-speci c subsets C = (C1; : : : ; Cn) based on a xed time granularity (e.g., week, month, or year). We then preprocess the documents of each subset Ci 2 C and train a Word2Vec Continuous Bag of Words (CBOW) model. Each model generates a matrix keys dimension(d) where keys represents a word from the corpus Ci for which a d-dimensional vector exists in the trained model.
To nd the relatedness score between a query entity eq 2 Eq and a candidate entity ec 2 Ec, we compute the cosine similarity of their word vectors in the corresponding trained model for the input time period Tq. In our experiments, for nding the words that represent a query or candidate entity, we use the last part of the entity's Wikipedia URI by rst replacing the underscore character with the space character and removing any text in parentheses. Based on the underlying knowledge base, one could use here other approaches, e.g., exploit the entity's label in DBpedia. For multi-word entities, we compute the average vector of the constituent words, while for inputs consisting of more than one entity, we compute the average of all entity vectors. 3.2
There are two limitations in dealing with the embedding vectors obtained in
the previous model: (i) for multi-word entities the average of the word vectors
of the constituent tokens is computed, i.e., n-grams are ignored (consider for
example the entity United Nations), and (ii) the same entity mention (surface
form) may refer to di erent entities (e.g., Kobe may refer to the basketball player
Kobe Bryant or the Japanese city ). To cope with these problems, we exploit the
entity annotations ED. Recall that each extracted entity is associated with a
unique URI in a knowledge base (e.g., Wikipedia or DBpedia). As also suggested
in [
Now, the similarity score between a query entity eq 2 Eq and a candidate entity ec 2 Ec is the cosine similarity of their word vectors using the corresponding modi ed (with entity IDs) Word2Vec model for the input time period Tq.
Formally:
Sim(eq; ec; Tq) = cosine(wordvec(id(eq); Tq); wordvec(id(ec); Tq)) (1) where id(e) is the ID of entity e, and wordvec(w; Tq) returns the word vector of token w using the corresponding model for the input time period Tq. 3.3
An important event related to the query entities may happened very close to the boundaries of the time period of interest Tq. This means that two entities might be highly related some time before or after Tq. To cope with this problem, the similarity score can also consider the Word2Vec models for the time periods before and after Tq. Let Tq 1 and Tq+1 be the time periods of granularity before and after Tq, respectively. The similarity score between a query entity eq 2 Eq and a candidate entity ec 2 Ec can now be de ned as follows: Sim0(eq; ec; Tq) = Sim(eq; ec; Tq) w1 +Sim(eq; ec; Tq 1) w2 +Sim(eq; ec; Tq+1) w3 (2) where w1, w2 and w3 are the weights of the models of Tq, Tq 1 and Tq+1, respectively, with w1 + w2 + w3 = 1:0.
This modi cation can increase the ranking of an important candidate entity that co-occurs frequently with a query entity some time before or after Tq, but at the same time can decrease the ranking of an entity co-occurring with a query entity during Tq. Thus, we should avoid using a very small w1 value. 4
Our objective is to evaluate the e ectiveness of the proposed approach and
compare it with similar but time and entity agnostic models. We use the dataset
and ground truth provided by [
We build 7 CBOW models using the proposed approach for each month from
Jul'14 to Jan'15 by considering only the articles in English and using the default
3 http://km.aifb.kit.edu/sites/ter/
Word2Vec setting: 300 dimensions, 5 words window size, 5 minimum word count
(as also used in [
Table 1 shows the results of normalized Discounted Cumulative Gain (nDCG)
[
Table 2 shows the e ect of relaxing the time boundaries for di erent w1 values, where w2 = w3. In general we see that the e ect of relaxing the time boundaries is very small in almost all cases. We notice that considering only the query time period Tq provides the best results in all cases apart from k = 5 where w1 = 0:9 slightly performs better. Moreover, we notice that as the value of w1 decreases (which means increased consideration of Tq 1 and Tq+1), the e ectiveness of the model gets worse. This is an expected result since, for the majority of the query entities in the dataset, the important event related to these entities did not happen very close to the time boundaries.
An example for which the relaxation of the time boundaries has a positive
impact on the ranking is for the entity 2014 FIFA World Cup. For w1 = 0:8,
nDCG@5 increases from 0.45 to 0.51. Notice that the time period of interest
for this entity is July 2014, however the tournament started on June 12, 2014.
Another example is the query entity Tim Cook (the CEO of Apple). For w1 = 0:8,
nDCG@5 increases from 0.58 to 0.62. Here the query time period is October 2014,
4 Note that our approach cannot be compared with [
We see that the relaxation of time boundaries can positively a ect entity relatedness when: i) an important event related to the query entity happened very close to the boundaries of the query time period, and ii) the query entity actually corresponds to an event which spans a long time period. Detecting such cases where time boundaries relaxation should be applied is beyond the scope of this paper but an important direction for future work. We have proposed a exible model for entity relatedness that considers timedependent and entity-aware word embeddings by exploiting the corpus of the underlying application. The results of a preliminary evaluation have shown that the proposed approach signi cantly outperforms similar but time and entityagnostic models.
As regards future work, an interesting direction is to extend the proposed
method for supporting arbitrary time intervals, which may require joining the
results from many models of smaller granularity. However, for supporting very
short time periods (e.g., day), this may also require the creation of a large number
of Word2Vec models. Regarding the relaxation of time boundaries, there is a need
of methods that can identify the most reasonable time window to consider given
the query entity and time period (by detecting, for example, periods of increased
entity popularity [
The work was partially funded by the European Commission for the ERC Advanced Grant ALEXANDRIA under grant No. 339233.