The extraction of named entities and their linking to a Semantic Web knowledge base is a task whose results support the process of extracting potentially useful elements of information from unstructured text. In the last decade, this task has been widely addressed with the aim of making possible the interconnection, exchange, and query of data on the Semantic Web. In this sense, several approaches based on the idea of ensemble methods (like those in Machine Learning) have been proposed to combine distinct named entity extraction and linking techniques in order to get better results than using a single such technique. Although the idea is to exploit features provided by diverse approaches to extract and link named entities from text, there are some issues to solve for integrating their results (e.g., heterogeneous output, duplicated entities). In this paper, we propose a strategy to integrate the output provided by some entity extraction and linking tools in an ensemble-like scheme. For such purpose, we consider steps for collecting and merging results supported by ltering decisions to overcome issues such as duplicated and/or overlapped entities. The results showed an increased performance in terms of the F-measure compared to isolated approaches.
The Semantic Web provides an extension of the Web in order to give a semantic
and formal representation of data, in such a way that the information could be
shared and reused by di erent applications [
On the other hand, in the eld of text mining, a named entity (NE) is an
important piece of information that refers to real or abstract things of the world,
such as names of persons, places, dates, among others. In the context of the
Semantic Web and following the LOD principles, a named entity has associated a
unique IRI, which is used to describe it in a Knowledge Base (KB). In knowledge
extraction, a common task is to look up for named entities and link them to a
KB. This process is known as Entity Extraction and Linking (EEL, a.k.a, named
entity disambiguation or entity linking) [
This paper presents a strategy to integrate the results provided by di erent EEL approaches to extract and link named entities from unstructured English text. The proposed strategy is composed of three stages to collect, merge and lter such results in order to increase the amount of extractions and reduce possible discrepancies and/or irrelevant extractions without drastically decrease the accuracy of the nal result. The proposed strategy di ers from existing EEL ensemble-based approaches in the sense that merging and ltering decisions are provided, which are helpful to mitigate the problems previously stated and to easily implement a prototype by leveraging available EEL tools and systems from the literature. Extractions provided by the proposed strategy would bene t applications in areas such as Ontology Learning, Machine Learning and Information Retrieval, to mention a few. Details of the proposed strategy are presented in the following sections. 2
The proposed strategy for extracting and linking named entities from unstructured text is based on the integration of the output provided by di erent EEL 1 http://wiki.dbpedia.org/ 2 https://www.mpi-inf.mpg.de/departments/databases-and-informationsystems/research/yago-naga/yago/ tools. This strategy consists of three main stages, as shown in Figure 1. These stages are described in the following subsections.
{JXSMOLN, , RDF
Entities <ME0, IRI0> <ME1, IRI1> {Overlaps
Duplicates The main purpose of this stage is the execution of di erent EEL tools taking into account that these tools can be executed in a local environment (algorithm execution) or remotely (invoking web services via HTTP requests). Thus, the EEL tools have to be already selected for this stage. In recent years, the most common way to make available an EEL tool is through web services, where the invocation of an EEL service is made through an HTTP request, sending one or more parameters like the input text, con dence, output format (e.g. JSON, XML, or RDF), among others to the server. The use of web services instead of standalone applications can be explained by several factors like computational requirements to execute the extraction algorithms, the size of dictionaries or knowledge bases (used to link entities) and to keep private some aspects of their internal processes. Hence, this rst stage collects the results from di erent EEL tools selected beforehand, keeping the original output from each tool. The result returned by each EEL tool has a prede ned number of features, which in most cases are di erent from one EEL tool to another. To manipulate all the results from the EEL tools it is necessary to merge them. This stage integrates the output from each used EEL tool taking into account the heterogeneity of features. For example, given the sentence Monterrey city is located in Mexico, the result returned by the DBpedia Spotlight3 web service for the named entity Monterrey is shown below: { "@text":"Monterrey city is located in Mexico", "@confidence":"0.35", "@support":"0", "@types":"", "@sparql":"", "@policy":"whitelist", "Resources":[
{ ]} }, { }, { "@URI":"http://dbpedia.org/resource/Monterrey", "@support":"2558", "@types": "Schema:Place,DBpedia:Place,DBpedia:PopulatedPlace,
DBpedia:Settlement,Schema:City,DBpedia:City", "@surfaceForm":"Monterrey", "@offset":"0", "@similarityScore":"0.9934778456173416", "@percentageOfSecondRank":"0.005987130977590879"
The DBpedia Spotlight service returns several features such as @URI, which refers to the identi er associated with the mention (the entity); @support refers 3 https://github.com/dbpedia-spotlight/dbpedia-spotlight to the degree of con dence of the resource assigned to the named entity in the KB; @surfaceForm refers to the text of the mention; @o set refers to the position of the mention in the input text (number of characters); @similarityScore refers to the similarity between the mention and the descriptive label of the resource in the KB; @percentageOfSecondRank indicates a degree of support relative to the ambiguity between possible resources allocated to the mention.
Although the names of the returned features vary for each EEL tool, there are common features such as the surface form (string of the entity mention), the position of the mention in the input text (o set), and the identi er (IRI) from the KB. Therefore, this step is intended to match the varied outputs produced by the selected EEL tools in such a way that the three previously mentioned features are obtained and thus, an homogeneous result can be produced. 2.3
The nal step lters the result to solve issues like duplicates and overlapped entities. For such purpose, three main cases are considered: 1. Duplicate entity mentions. This case occurs when two or more tuples have the same text as entity mention, but di erent identi er. In this case, a majority voting strategy was implemented to select the most frequent tuple. In the case of a tie, the EEL tools can be manually ranked so that the entity returned by the best ranked tool is selected. 2. Overlap entities. It refers to the case when an entity mention is partially contained or overlapped within another mention. For example, the entity mention Police Department is partially contained in the mention Monterrey Police Department. The proposed strategy is to keep the longest matching named entity with respect to the input text. 3. Duplicate tuples. For this case, duplicate tuples are eliminated, preserving one of the tuples returned by either of the invoked tools.
Finally, the result is stored in JSON format to be used by any application. 3
This section presents the experiments and results conducted for the proposed strategy. For such purposes, a Java implementation was developed and con gured as presented in the following subsections. 3.1
The experiments were performed using the testing dataset provided in the Open Knowledge Extraction Challenge (OKE)4 event. This dataset contains 193 manually labelled sentences with 875 tuples of entities linked to DBpedia. Note that NIL (Not In Lexicon) entities are not considered for these experiments (those entities that are relevant but do not contain an association with a KB resource). 4 https://github.com/anuzzolese/oke-challenge-2016
Three state of the art EEL tools were employed for the experiments. All of them
were invoked via web service: DBpedia Spotlight, Babelfy5 and TagMe6. These
tools were selected because they provide con gurable and publicly available web
services, additionally, they have shown top results as reported in [
Each tool was con gured using di erent con dence values in order to obtain a balance between the number of extracted entities and precision. For DBpedia Spotlight the con dence value was set as 0.5, for TagMe as 0.07 and for Babelfy as 0.5. Finally, traditional information retrieval metrics (precision, recall and Fmeasure (F1)) were used to evaluate the performance of the proposed strategy following an exact matching comparison, i.e., the extracted entity tuples must match exactly to those provided in the testing dataset. The results are shown in Table 1, where the results of each individual EEL tool are compared with respect to the results provided by the proposed integration strategy.
EEL system Precision Recall
F1 DBpedia Spotlight Babelfy TagMe Proposed strategy
As shows Table 1, the results of each individual EEL tool are not the best for Precision, Recall and F1 simultaneously. Meanwhile, the proposed strategy obtained the best values for Recall and F1. Moreover, an additional number of entities were extracted than using any single tool.
The proportion of true positives (TP), false positives (FP) and false negatives (FN) obtained by each tool for the testing dataset are shown in Table 2. These values were used to obtain the values depicted in Table 1. 3.3
The results presented in the experiments show that the proposed strategy increases the performance with respect to the F1 measure in comparison with the tested EEL tools. However, given that the result provided by the tools is merged into one homogeneous output, the Precision is negatively a ected since 5 http://babelfy.org/ 6 http://tagme.di.unipi.it/ EEL system TP FP FN the False Positive values are accumulated for the nal result, which directly a ect the Precision of the proposed approach. On the other hand, the output integration produced a greater number of entities linked to DBpedia from unstructured text. In some cases, False Positive named entities were extracted because the testing dataset does not have a link to describe them. For example, given the sentence \In 1842, a woman graduated with distinction from the National Autonomous University of Mexico", the proposed approach extracted the mention \woman", linked it to the resource http://dbpedia.org/resource/Woman, this result is marked as a false positive because this tuple does not appear in the testing dataset. 4
This paper presented a strategy to extract and link named entities from unstructured text. The strategy is based on the integration of three di erent state of the art EEL tools and the addition of three ltering rules to tackle problems of duplicated and overlapped entity mentions. The results obtained in the experiments demonstrate that the proposed strategy improves the extraction and linking according to the F1 measure and, additionally, the proposed strategy increases the number of entities linked to DBpedia KB in comparison with each EEL tool tested at the cost of a slight impact in the nal precision.