We report on our experiments for the CLEF 2013 Entity Recognition Challenge. Our approach is based on a combination of machine translation and NE tagging techniques. The Silver Standard Corpus (SSC) is used to obtain a corresponding annotated corpus in the target language. The plain text of the SSC is translated and a mapping is created between entities in the original and phrases in the translation, to which are associated the same CUIs as in the original. This produces a Bronze Standard Corpus (BSC) in the target language. A dictionary of entities is also created, which associates to each pair (entity text, semantic group) the corresponding CUIs that appeared in the SSC. The BSC is used to train a model for a Named Entity tagger. The model is used for tagging entities in sentences in the target language with the proper semantic group and the entity dictionary is used for associating CUIs to each of them.
1. a terminological resource (TR) produced from UMLS, containing English and non
English concepts together with their CUIs. 2. a selection of corpora in English, i.e. patent texts, Medline titles and EMEA documents, where the entity mentions have been annotated automatically with their CUIs. 3. a selection of corpora in different languages other than English, i.e. in DE, FR, SP, and NL, that have to be annotated with entity mentions and their CUIs. The English corpora have been annotated automatically with the help of several annotation solutions (for English) from the project partners and a harmonisation scheme to generate a Silver Standard Corpus (CALBC approach). The English corpus thus serves as additional input, but does not serve as a Gold Standard. 2
We designed our system while we did not have yet access to the UMLS TR resources provided by the organizers, therefore we just exploited the annotated English Silver Standard Corpus (SSC) as a source of information. We will discuss later how the TR resources could be integrated in our approach.
Our approach combines techniques of machine translation with NER techniques.
The following is an overview of the approach: 1. we apply phrase-based statistical machine translation to the SSC in order to obtain a corresponding annotated corpus in the target language. The plain text of the SSC is translated and a mapping is created between entities in the original and phrases in the translation, to which are associated the same CUIs as in the original. This produces a Bronze Standard Corpus (BSC) in the target language. A dictionary of entities is also created, which associates to each pair (entity text, semantic group) all the corresponding CUIs that appeared in the SSC. 2. the BSC is used to train a model for a Named Entity tagger, whose output classes are the possible semantic groups of entities. 3. the model built at step 2) is used for tagging entities in sentences in the target language with the proper semantic group. 4. the annotated document is converted to XML format and enriched by adding CUIs to each entity, looking up the pair (entity, group) in the dictionary of CUIs built in step 1.
One advantage of this approach is that it produces data, a NER model and an entity dictionary, that can be readily applied to a document in the target language without any further reference to the source corpora in the original language. 2.1
The Bronze Standard Corpus is obtained by translating the SSC and transferring to it the entity annotations.
For translating the original English SSC into the target language (Spanish), we use
Moses [
We exploited the word alignment information produced by Moses to determine the correspondence between entities in the source and target sentences.
Moses was trained on texts pertaining to the biomedical domain, obtained by
joining the EMEA corpus [
In our experiments we only dealt with English to Spanish translation, but the approach can be applied to any language pair for which there exist suitable parallel corpora.
In order to identify in the target language the phrases that correspond to entities in the original, we exploit the word alignment information obtained by invoking the Moses decoder with the option:
-alignment-output-file file Let’s illustrate this step with the following example.
“This medicine/CHEM relieves headaches/DISO” By invoking the Moses decoder, we obtain the following best translation: “Este medicamento alivia los dolores de cabeza” and the following word alignment:
1-1 2-2 3-3 3-4 3-5 3-6 Each pair of numbers in the alignment provides a correspondence between a token in the original sentence identified by its left number position and a token in its translation identified by its right number position.
The word alignment allows us to map an entity in the source to its translation.
For example, the entity “medicine”, located at position ‘1’ in the original sentence, is mapped to the single word “medicamento”, at position ‘1’ in the translation.
The case is less clear for the second entity: the word alignment indicates that the
source word “headaches”, located at position ‘3’, maps to the list of tokens [
In the example, article “los” gets dropped from the beginning, producing the final
entity “dolor de cabeza”. This step is performed by simply checking the part of speech
tags of tokens, obtained by using the Tanl POS tagger [
The training set for the NE tagger is obtained from the BSC, converting it into IOB notation and adding POS tags to each token.
The example in previous section would be represented as follows:
For performing NE recognition, we used the Tanl Tagger [
The Tagger can be configured to use alternative types of classifiers: Maximum
Entropy or Linear Support Vector [
For training moses we used a parallel corpus consisting of 247,655 sentences from the
English-Spanish version of Medline and 1,098,333 sentences from the EMEA corpus
[
We ran the training of Moses, using the KenLM [
Word alignment was done using MGIZA, a multi thread version of GIZA++ [
The Moses decoder was run using this model with default settings except for a beam size of 500. 3.3
The Tanl NE tagger can be customized by specifying the set of features to use.
Features are divided into local and global features. Local features include attribute
features, extracted from attributes (e.g. Form, PoS, NE) of tokens in the vicinity of
current token, and morphological features, binary features extracted from a token
matching a given regular expression. For CLEFER we did not use any global feature:
properties holding at the document level. We tested several configurations, starting from
the one that proved best for Italian at Evalita 2011 [
Attribute We submitted three runs for evaluation. The official submission (EMEA_es_man.LR5.xml) identified 417,390 entities in the 140,552 units present in the Spanish test corpus EMEA_es_man.xml.
Some of the identified entities appear without the corresponding CUI while others appear with a very large number of CUIs. The former occurs typically for entities with a general meaning, that during translation get associated to several different original English words in the entity dictionary.
After the submission, we were able to address this problem by performing a cleanup of the entity dictionary, in this way: for each pair (e, cl) in the dictionary for each c in cl retrieve the set of text entities te associated to c in the UMLS for target language if e is not present in te remove it from cl 5
We reported our experiments in the CLEF-ER Challenge 2013 for English to Spanish. By exploiting phrase-base machine translation and NE tagging we were able to build a system that can operate standalone, on any text in the target language, effectively transferring the knowledge on entities from one language to another.
The accuracy of the solution can be further refined by making use of data from the UMLS, as we started doing in our latest experiments.
Acknowledgements. Partial support for this work was provided by project RIS (POR RIS of the Regione Toscana, CUP n° 6408.30122011.026000160).