In this paper we present an approach for extracting and linking entities from short and noisy microblog posts. We describe a diverse set of approaches based on the Semanticizer, an open-source entity linking framework developed at the University of Amsterdam, adapted to the task of the #Microposts2015 challenge. We consider alternatives for dealing with ambiguity that can help in the named entity extraction and linking processes. We retrieve entity candidates from multiple sources and process them in a four-step pipeline. Results show that we correctly manage to identify entity mentions (our best run attains an F1 score of 0.809 in terms of the strong mention match metric), but subsequent steps prove to be more challenging for our approach.
This paper describes our participation in the named entity extraction and linking challenge at #Microposts2015. Information extraction from microblog posts is an emerging research area which presents a series of problems for the natural language processing community due to the shortness, informality and noisy lexical nature of the content. Extracting entities from tweets is a complex process typically performed in a sequential fashion. As a first step, named entity recognition (NER) aims to detect mentions that refer to entities, e.g., names of people, locations, organizations or products (also known as entity detection), and subsequently to classify the mentions into predefined categories (entity typing). After NER, named entity linking (NEL) is performed: linking the identified mentions to entries in a knowledge base (KB). Due to its richness in semantic content and coverage, Wikipedia is a commonly used KB for linking mentions to entities, or deciding when a mention refers to an entity that is not in the KB, in which case it is referenced by a NIL identifier. DBpedia aims to extract structured information from Wikipedia, and combines this information into a huge, cross-domain knowledge graph which provides explicit structure between concepts and the relations among them.
Permission to make digital or hard copies of all or part of this work for pCeorpsoynriaglhotr cclas2s0ro1o5mheulsde bisygaraunttheodr(wsi)t/howutnfeere(sp);rocvoidpeydintghapt ecrompiettseadre noontlymfaodreporrivdaistteriabnutdedacfoardepmroifict pour rcpoomsemse.rcial advantage and that copies bPeuabrltihsihsendoatiscepaanrdt othfetfhuell#ciMtatiicornopoonstthse20fi1r5st Wpaogrek. sThoopcoppryoocteheedriwnigsse,, to raevpauilbalbislhe, otonlpionset aosn CseErvUerRs oVrotlo-1r3ed9i5st(rhibtuttpe:t/o/lciestusr,-rwesq.uoirregs/pVroilo-r1s3p9e5c)ific permission and/or a fee. #Microposts2015, May 18th, 2015, Florence, Italy.
WWW #Microposts2015 at WWW ’15, 18-22 May 2015, Florence, Italy Copyright 2015 ACM X-XXXXX-XX-X/XX/XX ...$15.00.
Our participation in this challenge revolves around the existing
open-source entity linking software developed at the University of
Amsterdam. We use Semanticizer1, a state-of-the-art entity linking
framework. So far Semanticizer has been successfully employed in
linking entities in search engine queries [
Our system processes each incoming tweet in four stages: mention detection, entity disambiguation and typing, NIL identification and clustering, and overlap resolution. We explain each stage in turn. Mention detection: The first step aims to identify all entity mentions in the input text, and is oriented towards high recall. We take the union of the output of two mention identification methods:
Semanticizer: the state-of-the-art system performs lexical matching of entities’ surface forms. These surface forms are derived from the KB, and comprise anchor texts that refer to Wikipedia pages, disambiguation and redirect pages, and page titles as described in Table 1. For this, we use two instances of Semanticizer, running on two Wikipedia dumps: one dated May 2014 (the version used to build DBpedia 3.9), and a more recent one, dated February 2015.
We perform three separate preprocessing steps on the tweet text, the results of which get sent to the Semanticizer. These steps are: i) the raw text, ii) the cleaned text (replacing @-mentions with corresponding Twitter account names, and splitting hashtags using dynamic programming), and iii) the normalized text (e.g., casefolding, removing diacritics).
NER: For identifying entity mentions that do not exist in Wikipedia,
i.e., out of KB entities, we employ a state-of-the-art named entity
recognizer, previously applied to finding mentions of emerging
entities on Twitter [
Given the candidate mentions identified by NER and Semanticizer, we include a binary feature to express whether the mention has been detected by both systems. For each mention we end up with the set of features described in Table 1 that we use in training a Random Forest classifier (using 100 trees and rebalancing the classes per tweet by modifying instance weights), to predict whether a candidate mention is an entity mention (actually refers to an entity). 1https://github.com/semanticize/semanticizer
linkOccCount no. of times mention appears as anchor text on Wikipedia linkDocCount no. of docs in which mention appears as anchor text occCount no. of times the mention appears on Wikipedia senseOccCount no. of times the mention is anchor to Wikipedia title senseDocCount no. of docs the mention is anchor to Wikipedia title priorProbability % of docs where anchor links to target Wikipedia title linkProbability % of docs where mention is anchor for a Wikipedia link senseProbability % of docs where mention links to target Wikipedia article isCommon the mention is found by both NER and Semanticizer
from the previous stage, the next step is to identify referenced entities. We retrieve the full list of candidate entities, extract features, and cast the disambiguation step of identifying the correct entity for a mention as a learning to rank problem.
Next to the features in Table 1, we use additional full-text search features. We index Wikipedia using ElasticSearch (ES), and issue the tweet as a query for candidate entities’ retrieval scores. We also retrieve the 10 most similar entities for each candidate, using a more like this query. Finally, we incorporate Wikipedia page view statistics2 from April 2014 as features. We use these features to train RankSVM to rank the entity candidates for each mention, and take the top ranked candidate as the entity to link. We map the entity to its DBpedia URI, and determine its type through a manual mapping of DBpedia classes to the #Microposts2015 taxonomy.
ranked entity is correct, or the mention refers to an out-of-KB entity, we compute meta-features based on the RankSVM classifier’s scores. We use these meta-features to train a Random Forest classifier for NIL detection. We cluster NILs by linking identical mentions to a single NIL identifier based on their surface forms.
Overlap resolution: Finally, we resolve all overlapping mentions that are output by the mention identification step. We create a graph of all non-overlapping mentions, and assign them their link score (non-linked mentions get a fixed score). We then find the highest scoring path through the graph using dynamic programming, and return the mentions of this path as our resolved list of mentions. Our submitted runs rely on this scheme and variations thereof. See Table 2 for an overview of the runs. We hypothesize that the Semanticizer will yield high entity recall, but low precision. Filtering the resulting candidates by senseProbability will increase precision. We expect the NER runs to be superior to Semanticizer or ES-only runs. Finally, we believe that combining the NER and Semanticizer outputs with additional candidates returned by ES will outperform all our other runs.
We evaluate our approach on the dev set consisting of 500 tweets
made available by the organizers [
NIL Run 1 Run 2 Run 3 Run 4 Run 5 Run 6 Run 7 Run 8 Run 9 Run 10 0.809 0.809 0.809 0.554 0.411 0.620 0.533 0.732 0.577 0.566 0.456 0.460 0.455 0.311 0.288 0.389 0.330 0.418 0.365 0.355 0.164 0.330 0.291 0.213 0.280 0.280 0.210 0.334 0.247 0.280 MC 4.
We have presented a system that performs entity mention detection, disambiguation and clustering on short and noisy text by drawing candidates from multiple sources and combining them. We observe that our simple NER and ES runs perform better than our more complex runs. We believe that more robust methods are needed to deal with the errors introduced at each step of the pipeline. For future work we plan on improving mention detection with additional Semanticizer features.
Acknowledgements. This research was partially supported by the Netherlands Organisation for Scientific Research (NWO) under project numbers 727.011.005, SEED and 640.006.013, DADAISM; Amsterdam Data Science, and the Dutch national program COMMIT.