We present a named-entity recognition (NER) system for parallel multilingual text. Our system handles three languages (i.e., English, French, and Spanish) and is tailored to the biomedical domain. For each language, we design a supervised knowledge-based CRF model with rich biomedical and general domain information. We use the sentence alignment of the parallel corpora, the word alignment generated by the GIZA++[8] tool, and Wikipedia-based word alignment in order to transfer system predictions made by individual language models to the remaining parallel languages. We re-train each individual language system using the transferred predictions and generate a nal enriched NER model for each language. The enriched system performs better than the initial system based on the predictions transferred from the other language systems. Each language model bene ts from the external knowledge extracted from biomedical and general domain resources.
Natural Language Processing (NLP) technologies can help extract structured
information from written text, determine relationships between concepts, or
perform bilingual translation. In the biomedical and clinical[
In this study, we present a system for extracting structured information from
multilingual biomedical corpora. We de ne the structured information of interest
as noun phrases (NPs) that fall under nine Uni ed Medical Language System
(UMLS)[
We present a NP extraction system for multilingual biomedical corpora that makes use of external knowledge sources and the common structure shared by the parallel languages. We analyze the errors performed by the NER system, and we discuss the contribution of the knowledge sources and the common language structure to system performance. 1.1
We focus on NPs that fall under nine semantic categories: anatomy (ANAT),
chemicals and drugs (CHEM), devices (DEVI), disorders (DISO), geographic
areas (GEOG), living beings (LIVB), objects (OBJC), phenomena (PHEN), and
physiology (PHYS). These semantic categories are groupings of semantic types
in the UMLS semantic network.[
The data used for the system development is made available by the
organizers of the 2013 CLEF-ER challenge[
For the Medline corpus, we used the Medical Subject Heading (MeSH) indexing and the UMLS Metathesaurus to automatically generate noun phrase annotations. The MEDLINE citations corresponding to the sentences (i.e., titles) in the Medline corpus have been manually assigned a set of MeSH indexing terms by professional indexers at the National Library of Medicine. MeSH main headings can be derived from this set and mapped to UMLS CUIs. We randomly selected 385 Medline sentences for each language and made use of the MeSH indexing-UMLS CUI relationship in order to generate Medline reference annotations. Speci cally, when a term associated to a CUI from the MeSH indexing can be found in the corresponding Medline sentence, we automatically create a reference annotation. We apply the same annotation process to the English, French, and Spanish versions of the Medline corpus.
The EMEA and Medline reference annotations were manually reviewed by a sole annotator, due to time constraints and lack of additional resources.
The number of unique annotations for EMEA and Medline are similar across languages (368 English annotations for EMEA and 401 for Medline, 364 French annotations for EMEA and 339 for Medline, and 368 Spanish annotations for EMEA and 303 for Medline). The EMEA corpus contains a signi cantly larger number of actual annotations due to numerous noun phrase duplicates (see Table 2). 2.3
We perform word alignment using the GIZA++ software.[
In addition to the GIZA++ alignment, we use word alignment from Wikipedia metadata. We rely on the fact that some English Wikipedia articles have direct correspondents in French and Spanish. We lter the English articles that have titles consisting of a single word (e.g., \Food", \Disease", \Nausea") and nd their corresponding foreign language article. The title of the English article and the title of the correspondent foreign language article represent a word alignment pair. Because Wikipedia word alignments are more precise, we overwrite the GIZA++ generated word alignments with a score lower than 0.5 with the Wikipedia generated word alignments.
Annotation count
English French Spanish ANAT MEMedElinAe 26022 6301 3288 CHEM MEMedElinAe 158586 55159 55782 DEVI MEMedElinAe 4130 63 110 DISO MEMedElinAe 1166588 412870 416754 GEOG MEMedElinAe 3145 191 1232 LIVB MEMedElinAe 47922 16302 13357 OBJC MEMedElinAe 1195 347 328 PHEN MEMedElinAe 727 320 310
PHYS MEMedElinAe 22792 1523 664
Total NPs MEMedElinAe 4446911 1328661 1334704 Total Unique NPs EMEA 368 364 368
Medline 401 339 303 Table 2. Annotations description per corpus type: annotation count, separated by language and semantic category Noun phrase identi cation Our system is designed based on the supervised CRF framework. The CRF model includes lexical features (the normalized form of the token), syntactic features (part-of-speech and parse tree information of each token), and knowledge-based features (information extracted from UMLS and Wikipedia). A nal set of features is generated based on the shared structure of the parallel languages. All feature sets are used in the nal CRF model.
In order to obtain the parse and POS information we use di erent resources
for each language: for English we use the Stanford parser,[
The Wikipedia knowledge-based features are dependent on the language and are extracted based on the respective language version of Wikipedia. We make use of the fact that Wikipedia articles are tagged with speci c categories. We map these Wikipedia categories to one of the nine UMLS categories of interest. Then, we identify the n-grams (n 3) that represent titles of Wikipedia articles and signal within the attribute vector whether an n-gram belongs to one of the nine categories of interest based on its Wikipedia categorization.
Using the lexical, syntactic, and knowledge-based features described above, we create a CRF model (the Initial model) for each language and label the training data using the relevant model. We further make use of the common structure shared by the parallel corpora and transfer the NP predictions across aligned bilingual sentence pairs. We hypothesize that some language models might predict NPs that other language models would miss, and by using sentence- and word-alignment information we can inform the other language models of the missed predictions. The CRF model containing the transferred predictions is referred to as the enriched model. The entire system design is depicted in Figure 1. The NER system predicts the text of the NPs together with their semantic categories. In order to obtain the CUI associated with each NP we use MetaMap for English and direct lookup inside the UMLS database for the French and Spanish languages. Because the French and Spanish versions of UMLS do not contain the same number of concepts as the English UMLS, there are French and Spanish concepts identi ed by our system that cannot be linked to a CUI via direct UMLS lookup. For those concepts we rely on word alignment to determine the English concept to which the foreign language concepts are aligned; once an aligned English concept is found we transfer the CUI information to the foreign language concept. 3
The reference annotations we generated for this challenge relied partially on automated annotation tools. These annotation tools generated an imperfect annotation output and generally failed to identify a percentage of the noun-phrase instances. For the Medline corpus, the annotation tools failed to identify NPs in the sentences that belonged to oldest citations. These citations were only assigned a couple of MeSH indexing terms, vs. about a dozen for more recent citations. Also, because the MeSH indexing terms are assigned based on the fulltext article content and not on the article title, they might be linked to CUIs not present in the title. A similar issue arises when the Medline title contains a shortened or altered version of the CUI string indexed by MeSH (e.g. the string \hypertensive patient" occurring in the title of citation 1838917 could not be reconciled with CUI C0020538 \hypertension"). For EMEA, the main problem was the transfer of English annotations based on the word alignment: the GIZA++ software does not generate a perfect word alignment, and even though the Wikipedia word alignment manages to x some incorrect alignments, there are still cases of incorrect or missing alignments.
The problem of the noisy word alignment impacts both the generation of the reference annotations and the performance of the enriched system. Speci cally, the predictions transferred between the systems are usually incomplete (e.g., for the English noun phrase active substance, the word alignment could only help transfer the labeling for the token \substance" and failed to transfer the labeling for the token \active" as it could not nd a valid word French or Spanish alignment for the word \active").
Mapping the noun phrases to a CUI is a di cult problem for the French and Spanish languages since the number of annotated CUIs is relatively small. A large number of UMLS English concepts are not present in the foreign language portions of UMLS; thus, even though the English translation of a French or Spanish noun phrase is present in UMLS, the French or Spanish noun phrase is not included. Our system has to rely on machine translation or noun phrase alignment for mapping certain French and Spanish noun phrases to their CUIs.
Based on manual revision, the noun phrases identi ed by the enriched model present a higher precision and slightly lower recall. In general, the enriched model predicts more complete noun phrases (e.g., \insu sance hepatique severe" vs. \insu sance hepatique", \diminution des bruits respiratoires", vs. \bruits respiratoires", \paresies des cordes vocales" vs. \des paresies des cordes", \tumor principal" vs. \tumor", \movimientos involuntarios" vs. \involuntarios"). We notice that the semantic categories of the noun phrases are better assigned when the enriched model is used (e.g., \linfocitos" assigned a LIVB category by the initial model and an ANAT category by the enriched model), and even correctly adjusted together with the span of the noun phrase (e.g., noun phrase \mand bula" classi ed as ANAT in the initial model is changed into osteonecrosis de la mandbula classi ed as DISO in the enriched model). 4
We present a multilingual NER extraction system targeting the biomedical domain. The NER system relies on a supervised learning framework enriched with external knowledge and cross-linguistic information transferred based on common structure between languages. We prepare reference annotations for English, French, and Spanish that together with the NER system can be used for developing additional multilingual resources. We illustrated the improvements brought by our second round of training based on cross-language transfer of initial annotations. 5
This work was funded in part by award number 2U54LM008748 from the National Institutes of Health (NIH)/National Library of Medicine (NLM) and cofounded by the National Heart, Lung and Blood Institute (NHLBI), and by contract number 90TR0002 (SHARPd Secondary Use of Clinical Data) from the O ce of the National Coordinator (ONC) for Health Information Technology. The main author was funded by the Chateaubriand Science Fellowship 2012-13.