This paper describes the participation of the UNIBA team in the Named Entity rEcognition and Linking (NEEL) Challenge. We propose a knowledge-based algorithm able to recognize and link named entities in English tweets. The approach combines the simple Lesk algorithm with information coming from both a distributional semantic model and usage frequency of Wikipedia concepts. The algorithm performs poorly in the entity recognition, while it achieves good results in the disambiguation step.
In this paper we describe our participation in the Named
Entity rEcognition and Linking (NEEL) Challenge [
We propose two approaches that share the same methodology to disambiguate entities, while die↵ring in the approach used to recognize entities in the tweet. We implement two algorithms for entity detection. The former (U N IBAsup) exploits PoS-tag information to detect a list of candidate entities, while the latter (U N IBAunsup) tries to find sequences of tokens (n-grams) that are titles of Wikipedia pages or surface forms which refer to Wikipedia pages.
The disambiguation and linking steps rely on a knowledgebased method that combines a Distributional Semantic Models (DSM) with the prior probability assigned to each DBpedia concept. A DSM represents words as points in a mathe
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are nCootpmyraidgehtor cdis2tr0ib1u5tehdelfdorbpyroafiuttohrocro(ms)m/oewrcniaelr(asd)v;acnotapgyeinagndptehramt ictotpedies boenalyrtfhoirs nportiivcaeteanadntdheafcualdl ecmitaictiopnuorpnothseesfi.rst page. To copy otherwise, to rPeupbublilsihshe,dtoaspopsatrotnosfertvheers#oMrtiocrroedpiosstrtisb2u0t1e5toWliostrsk,srheqoupirpersopcreieodrisnpgesc,ific paveramilaisbslieononanlidn/eoraas fCeeE.UR Vol-1395 (http://ceur-ws.org/Vol-1395) W#WMWicroposts2015, May 18th, 2M01a5k,inFgloSreenncsee, oIftaMlyi.croposts (#Microp2015 - #Microposts2015 osts2015) - NEEL Challenge Copyright 20XX ACM X-XXXXX-XX-X/XX/XX ...$15.00. matical space; words represented close in this space are similar. The word space is built analyzing word co-occurrences in a large corpus. Our algorithm is able to disambiguate an entity by computing the similarity between the context and the glosses associated with all possible entity concepts. Such similarity is computed through the vector similarity in the DSM. Section 2 provides details about the adopted strategies for: 1) Entity Recognition and 2) Linking. The experimental evaluation, along with commentary about results, are presented in Section 3. 2.
Our methodology is a two-step algorithm consisting in an initial identification of all possible entities mentioned in a tweet followed by the linking (disambiguation) of entities through the disambiguation algorithm. DBpedia is exploited twice in order to 1) extract all the possible surface forms related to entities, and 2) retrieve glosses used in the disambiguation process. In this case we use as gloss the extended abstract assigned to each DBpedia concept. 2.1
In order to speed up the entity recognition step we build an index where each surface form (entity) is paired with the set of all its possible DBpedia concepts. The index is built by exploiting Lucene API2, specifically for each surface form (lexeme) occurring as the title of a DBpedia concept3, a document composed of two fields is created. The first field stores the surface form, while the second one contains the list of all possible DBpedia concepts that refer to the surface form in the first field. The entity recognition module exploits this index in order to find entities in a tweet. Given a tweet, the module performs the following steps: 1) Tokenizing and PoS-tagging the tweet via Tweet NLP4; 2) Building a list of candidate entity. We exploit two approaches: all n-grams up to five words (U N IBAunsup); all sequences of tokens tagged as proper nouns by the PoS tagger (U N IBAsup); 3) Querying the index and retrieving the list of the top 25 matching surface forms for each candidate entity; 4) Scoring each surface form as the linear combination of: a) the score provided by the search engine; b) a string similar2http://lucene.apache.org/ 3We extend the list of possible surface forms using also the resource available at: http://wifo5-04.informatik. uni-mannheim.de/downloads/datasets/ 4http://www.ark.cs.cmu.edu/TweetNLP/ ity function based on the Levenshtein Distance between the candidate entity and the surface form in the index; c) the Jaccard Index in terms of common words between the candidate entity and the surface form in the index; 5) Filtering the candidate entities recognized in the previous steps: entities are removed if the score computed in the previous step is below a given threshold. In this scenario we set the threshold to 0.85. The output of the entity recognition module is a list of candidate entities in which a set of possible DBpedia concepts is assigned to each surface form in the list. 2.2
We exploit an adaptation of the distributional Lesk
algorithm proposed by Basile et al. [
1. Building the glosses. We retrieve the set Ci = {ci1, ci2, . . . , cik} of DBpedia concepts associated to the entity ei. For each concept cij , the algorithm builds the gloss representation gij by retrieving the extended abstract from DBpedia. 2. Building the context. The context T for the entity ei is represented by all the words that occur in the tweet except for the surface form of the entity. 3. Building the vector representations. The context T and each gloss gij are represented as vectors (using the vector sum) in the DSM. 4. Sense ranking. The algorithm computes the cosine similarity between the vector representation of each extended gloss gij and that of the context T . Then, the cosine similarity is linearly combined with a function that takes into account the usage of the DBpedia concepts. We analyse a function that computes the probability assigned to each DBpedia concept given a candidate entity. The probability of a concept cij is computed as the number of times the entity ei is tagged with the concept cij in Wikipedia. Zero probabilities are avoided by introducing an additive (Laplace) smoothing.
We exploit the rdf:type relation in DBpedia to map each DBpedia concepts to the types defined in the task. In particular, we provide a manual map for all the types defined in the dbpedia-owl ontology to the respective types provided
6We use 400 dimensions for vectors analysing only terms that occur at least 25 times. by the organizers. 3.
This section reports results of our system on the development set provided by the organizers. The dataset consists of 500 manually annotated tweets. Results are reported in Table 1. The first column shows the entity recognition strategy, the other columns report respectively the F-measure of: strong link match (SLM), strong typed mention match (STMM), mention ceaf (MC). SLM measures the linking performance, while STMM takes into account both link and type. MC measures both recognition and classification.
U N IBAsup U N IBAunsup F-SLM 0.362 0.258
0.267 0.191
F-MC 0.389 0.306
We cannot discuss the quality of the overall performance since we have not information about both baseline and other participants. However, we can observe that the recognition method based on PoS-tags obtains the best performance. We performed an additional evaluation in which we removed the entity recognition module and took entities directly from the gold standard. The idea is to evaluate only the linking step. Results of this evaluation are very encouraging, we obtain a F-SLM=0.563, while excluding the NIL instances we achieve a link match of 0.825. These results prove the ee↵ctiveness of the proposed disambiguation approach based on DSM.
This work fulfils the research objectives of the PON project EFFEDIL (PON 02 00323 2938699). The computational work has been executed on the IT resources made available by two PON projects financed by the MIUR: ReCaS (PONa3 00052) and PRISMA (PON04a2 A).