We present an experimental study of the performance of a hybrid semantic and linguistic system for recognizing and linking entities from formal and informal texts. In the current literature, systems are generally tailored to one or a few types of textual documents (e.g. narrative texts, newswire articles, informal text such as microposts). In contrast, we assess the performance of a hybrid approach that adapts the entity extraction, recognition and linking process to the type of document being analyzed. The hybrid system relies on POS taggers, gazetteers and Twitter user account dereferencing modules to extract entities, NER modules to recognize and type entities and entity popularity, string distance measures and scoring functions to disambiguate entities by ranking potential link candidates. The evaluation results show the robustness of our proposed approach in terms of text-independence compared to the current state-of-the art.
The approach we analyze in this paper links entities from both formal and informal textual documents to DBpedia 2014 resources. The approach can be broken down into three tasks: entity extraction, entity recognition and entity linking. Entity extraction refers to the task of spotting mentions that can be entities in the text. Entity recognition refers to the task of giving a type to the extracted entity. Entity linking refers to the task of linking the mention to a targeted knowledge base, and it is often composed of two sub-tasks: generating candidates and ranking them according to scoring functions. We experiment with three standard benchmark corpora: two composed of tweets and one composed of formal texts.
Numerous approaches have been proposed to address the task of extracting, typing
and linking entities in formal (such as newswire content) and informal (such as
microposts) text. Among the recent and best performing systems, both WAT [
Our goal is to link all the entities occurring in a text to their counterparts in DBpedia 2014. Entities that do not have an entry in the knowledge base will be linked to NIL. Tweets are notoriously problematic to process compared to formal texts because of i) hashtags (such as #barackobama referring to Barack Obama); ii) user mentions (e.g. @ryeong9 referring to db:Ryeo Wook1); iii) acronyms (e.g. Met for Metropolitan Police Service); iv) short length of only 140 characters and v) syntax which is often grammar free and words are misspelled. In the following, we briefly describe the different components of our hybrid approach.
Text Normalization. Only for tweets, this step consists of normalizing the text. We remove emoticons, extra white spaces and punctuation symbols belonging to two unicode categories2: other and symbol.
Entity Extraction and Recognition. This step is about detecting mentions from text that are likely to be selected as entities. The objective is achieved by using the following components: 1) POS tagger; 2) NER; 3) gazetteer and 4) time expressions spotter. For tweets, we add a dereferencing Twitter account system to retrieve the user name. Our POS tagging system is the Stanford NLP POS Tagger with a model trained specifically for tagging tweets3 in order to be case insensitive and to get independent tags for mentions and hashtags. For formal text we use the english-bidirectional-distsim that provides a better precision but for a higher computing time. With the POS tagger, we spot all the noun phrases and numbers. As NER system, we use Stanford NER properly trained with the training set of a benchmark. The gazetteers reinforce this stage bringing a robust spotting for well-known nouns. Tweets contain Twitter accounts, so we dereference Twitter accounts and extracts their public names. Each of those systems are launched in parallel. The type of an entity is given by Stanford NER. If a mention is not detected by Stanford NER then no type is assigned.
Entity Resolution. Given two overlapping mentions, e.g. States of America from Stanford NER and United States from Stanford POS tagger, we only take the union of the two phrases. We obtain the mention United States of America and the type provided by Stanford NER is selected.
Entity Linking. This step is composed of three sub-tasks: i) entity generation,
where we lookup up the entity in an index built on top of both DBpedia20144 and a
dump of the Wikipedia articles5 dated from October 2014 to get possible candidates; ii)
candidates filtering based on direct inbound and outbound links between the extracted
entities in Wikipedia; iii) entity ranking based on an in-house ranking function using
Levenshtein distance between the extracted mention and the title, the set of redirect
pages and the set of disambiguation pages of each candidate, weighted by their
PageRank. If an entity does not have an entry in the knowledge base, we normally link it to
1 db stands for http://dbpedia.org/resource/
2 http://www.fileformat.info/info/unicode/category/index.htm
3 https://gate.ac.uk/wiki/twitter-postagger.html
4 http://wiki.dbpedia.org/services-resources/datasets/
datasets2014
5 https://dumps.wikimedia.org/enwiki/
NIL. The detailed method followed for the linking step and how to build the index is
explained in [
Our hybrid approach has been tested against the test dataset of the #Micropost2014 [
Datasets
OKE2015 #Microposts2014 #Microposts2015 co-references typing NIL entities dates numbers 3 3 3 7 7 7 7 7 3 3 7 3 3 7 7
Table 2. Features for each datasets The results that are reported for DBpedia Spotlight, TagME, AIDA and Babelfy for OKE2015 have been obtained using their respective APIs with the best tested settings. For DBpedia Spotlight, those settings are: confidence=0.3 and support=20. For TagME, those settings are: include all spots=yes and epsilon=0.5. For AIDA those settings are: technique=GRAPH, algorithm=COCKTAIL PARTY SIZE CONSTRAINED, alpha=0.6 and coherence=0.9. For Babelfy, those settings are: lang=en, annType=NAMED ENTITIES, annRes=WIKI, match=EXACT MATCHING, dens=true and th=0.4. Since WAT is not publicly accessible, we did not test it with the OKE challenge dataset. For formal text (OKE benchmark), our system outperforms the other approaches 6 https://github.com/anuzzolese/oke-challenge 7 http://multimediasemantics.github.io/adel/ 8 https://github.com/wikilinks/neleval being tested. For informal text (NEEL corpora), the hybrid approach shows the robustness in extracting and typing entities because we jointly use linguistic and semantic methods. The results slightly drop at linking level. 4
In this paper, we have presented the experimental study of a text independent hybrid approach that extracts, recognizes and links entities to DBpedia 2014. We show that a successful approach exploits both linguistic features and semantic features extracted from DBpedia. As future work, we plan to focus on improving the linking task by making more use of graph-based algorithms, and to improve our ranking function.
This work was partially supported by the innovation activity 3cixty (14523) of EIT Digital (https://www.eitdigital.eu).