This paper presents our system for the “Making Sense of Microposts 2014 (#Microposts2014)” challenge. Our system is based on AIDA, an existing system that links entity mentions in natural language text to their corresponding canonical entities in a knowledge base (KB). AIDA collectively exploits the prominence of entities, contextual similarities, and coherence to effectively disambiguate entity mentions. The system was originally developed for clean and well-structured text (e. g. news articles). We adapt it for microposts, specifically tweets, with special focus on the named entity recognition and the entity candidate lookup.
Named Entity Recognition. AIDA originally uses Stanford NER, with a model trained on newswire snippets, a perfect fit for news texts. However, it is not optimized for handling user generated content with typos and abbreviations. Hence, we employ two different components for mention detection: The first is Stanford NER with models trained for caseless mention detection; the second is our in-house dictionary-based NER tool. The dictionary-based NER is performed in two stages: 1. Detection of named entity candidates using dictionaries of all names of all entities in our knowledge base, using partial prefix-matches for lookups to allow for shortening of names or little differences in the later part of a name. For example, we would recognize the ticker symbol “GOOG” even though our dictionary only contains “Google”. The character-wise matching of all names of entities in our KB is efficiently implemented using a prefix-tree data structure. 2. The large number of false positives are filtered using a collection of heuristics, e. g. the phrase has to contain a NNP tag or it has to end with a suffix signifying a name such as “Ave” in “Fifth Ave”.
Mention Normalization. The original AIDA did not distinguish between the textual representation of the mention, and its normalized form that should be used to query the dictionary. For example, the hashtag "#BarackObama" should be normalized to “Barack Obama” before matching it against the dictionary. Furthermore, many mentions of named entities are referred to in the tweet by their Twitter user ID, such as "@EmWatson" the Twitter account of the British actress “Emma Watson”. Because the Twitter user IDs are not always informative we access the account metadata, which contains the full user name most of the time. In fact, we attach to each mention string a set of normalized mentions and use all of them to query the dictionary. For example "@EmWatson" will have the following normalized mentions {“EmWatson”, “Em Watson”, “Emma Watson”}, and each of them will be matched against the dictionary to retrieve the set of candidate entities. As the prior probability is on a per-mention basis, we compute the aggregate prior probability of an entity ei given a mention mi: prior(mi, ei) =
max m0∈N(mi) (1) where N (mi) is the set of normalized mentions of mi. The maximum is taken in order not to penalize an entity if one of the normalized mentions is rarely used to refer to it. Approximate Matching. This step is employed iff the previous normalization step did not produce candidate entities for a given mention. For example, it is not trivial to automatically split a hashtag like "#londonriots", and hence its normalized mention set, {”londonriots”}, does not have any candidate entities. We address this by representing both the mention strings and dictionary keys as vectors of charactertrigrams between which the cosine similarity is computed. We only consider the candidate entity if cosine similarity between the mention and candidate entity keys is above a certain threshold (experimentally determined as 0.6). Parameter Settings. In our graph representation, the weight of a mention-entity edge is computed by a linear combination of different similarity measures. To estimate the constants of the linear combination, we split the provided tweets training dataset into TRAIN and DEVELOP chunks, using TRAIN for the estimation. We estimated further hyper-parameters for our algorithm (like the importance of mention-entity vs. entityentity edges) on DEVELOP.
Unlinkable Mentions. Some mentions should not be
disambiguated to a entity, even though there are candidates
for it. This is especially frequent in the case of social
media, where a large number of user names are ambiguous but
do not refer to any existing KB entity – imagine how many
Will Smiths exists besides the famous actor. We address this
problem by thresholding on the disambiguation confidence
as defined in [
We conducted our experiments on the dataset provided in
[
During our experiments, our runs achieved around 51% F1 on the DEVELOP part of the training data, where a mention is counted as true positive only if both the mention span matches the ground truth perfectly and the entity label is correct.
AIDA is a robust framework that can be adapted to any type of natural language text, here we use it to disambiguate names to entities in tweets. We found that using a dictionary-based NER worked well for the sometimes illformatted inputs. An approximate candidate lookup crucially improves recall, which in combination with discarding low-confidence mentions improves the results.