The amount of abbreviations used in biomedical literature increases constantly. Despite the existence of acronym dictionaries, it is not viable to keep them updated with new creations. Thus, in the processing of biomedical texts, discovering and disambiguating acronyms and their expanded forms are essential aspects and this is the objective proposed by BARR task at IberEval 2017 Workshop. This paper presents our participation in this task. We propose ve systems that deal with the problem in di erent ways. Three of the systems are atomic approaches, while two of them are combinations of the atomic systems. One of the systems clearly outperforms the others, both in the detection of entities (F-score of 0.749 in the test set) as well as identifying relations between short-long forms (F-score of 0.697 in the test set).
The volume of biomedical texts is greater and greater, and at the same time,
the number of biomedical abbreviations is growing rapidly, being the
ambiguity of biomedical abbreviations a challenge. Particularly, handling abbreviations
without nearby de nitions is a critical issue [
The acronyms have a high reference value, in the sense that they most of
the time act as reference anchors of textual context [
Usually, existing work on acronym recognition in medical domain is proposed for English biomedical documents, being di cult to adapt these proposals to other languages. The BARR track has the aim to promote the development and evaluation of biomedical abbreviation identi cation systems in Spanish biomedical documents.
In this paper we present the approaches proposed by our team, in particular ve di erent proposals.
The remainder of this paper is organized as follows. Section 2 presents the proposed systems. Section 3 summarizes the results and discuss about them. Finally, conclusions are presented in Section 4. 2
We propose di erent approaches to identify entities, both short and long forms, in the texts and also the relations between them. Our team is composed by members of di erent universities, in a way that we propose a method by each university (URJC and UNED propose one method together) and two additional methods which combine the other three proposal in some way. Following we describe all the methods. 2.1
This system tries to take advantage of an already developed linguistic analyser,
called FreelingMed [
For the detection of entities, three main approaches are considered: i) an heuristic that marks word-forms that follow certain pattern usually appearing in abbreviations and acronyms; ii) elements that come marked as abbreviation or acronym from the dictionaries of FreelingMed; and, iii) elements that come from Freeling (in the basis of FreelingMed) marked as abbreviations referring to units of weight (e.g. mg ), length (e.g. cm), time (e.g. min) etc. 2.2
The second atomic system is based on the combination of three acronym /
expansion pair extractors covering roughly the three most frequent cases of acronym
/ expansion mentions:
{ Similarity-based. This approach tries to detect in a document (within the
title and the abstract) mentions of single words or multiwords likely to be an
acronym (short form) and an expansion (long form) so that the two forms are
likely able to be mapped using a set of 13 hand crafted mapping rules. These
mapping rules are applied in decreasing order of con dence. For recognizing
the short forms we have used the set of regular expressions proposed in [
This system combines a pattern-based approach with a dictionary-based approach, and consists on two steps: abbreviations detection and de nition matching for them.
In the rst step, we detect terms in capital letters or combination of capital
letters with lowercased letters, numbers and other characters. We use
parenthetical constructions as indicator of a possible abbreviation [
In addition, some special cases for the approach based on patterns are considered. For instance, the following text has two relation pairs for the same acronym and two di erent de nitions, one per language: \amino-terminal propeptide of procollagen type 1 (P1NP, propeptido aminoterminal del procolageno 1)".
In case of the pattern-based approach does not nd a valid de nition for the abbreviation, we use a dictionary where each entry is an abbreviation and its possible long forms. In the same order that long forms appear in the dictionary, we search each one in the same sentence where the abbreviation is, and the rst one that matches is selected as the long form of the pair of the relation. The dictionary used has 7,916 entries. 2.4
We have implemented three simple combination mechanisms named as and, or, and vot, that are applied over the results of the atomic systems. and accepts an annotation only in the case all three atomic systems propose it. or accepts all the annotations of the atomic systems just checking that no contradictions (partial overlapping) occur in the mentions. vot implements a democratic votation schema, i.e. an annotation is accepted in the case at least two of the atomic systems have proposed it.
For our nal submission only and and or combinations were submitted. 3
In this section we present the results obtained identifying entities and relations abbreviation-de nition.
The evaluation metric used for evaluating the participating systems of the
BARR track has been the F-score micro measure. The organization has provided
an adaptation of the Markyt platform for the evaluation [
The organization has provided training, test and background collections [
The UNED system obtains high precision values, specially identifying entities. This con rm that an approach based on patterns is suitable for this problem. The recall values are a bit lower due to this system does not detect nested entities. Moreover, it is probably the patterns did not detect all special cases that could appear in texts. The system considers some special cases which implies variations in the patterns, but it is possible that exist more special cases not considered.
The F-score is lower identifying relations because not for all entities detected the system nds a valid long form. The system searches long forms in a maximum number of words on the left of an acronym and it can be out of this window.
The main objective of the EHU system has been to reuse a linguistic analyzer that was already developed. This approximation is limited as it only can detect abbreviations already gathered in the dictionaries of FreelingMed and analyzed as an unique element in its long form. Table 1 shows in its relation column that a recall of 0.1 is obtained but with a precision of 0.9. Those results, in our opinion, are clearly related to the type of approach.
The results of UPC system were bad. There are several explanations for this: { The Gazetteer-based component had a very small contribution to the global system. { The acronym detector resulted in the training phase on many failures (about 50 false negatives and more than 200 false positive). Specially in the case of one character abbreviations the results were bad. { Also the detection of long form candidates resulted in many false positives, specially single word terms and multiterms starting or ending with a non valid POS. { Finally, some of the mapping rules, specially those involving a single word long form, presented a low accuracy.
Table 2 presents the results over the test set. For the test set only the and and or combinations were done between the UNED and UPC systems output.
Due to the big volume of data in the test set, only the 22.5 % of the corpus was analyzed in time in the EHU approach and there were some computer memory problems in the UPC approach. FreelingMed is quite slow (39 sec. to analyze an abstract of 178 tokens) due to the volume of the dictionaries it uses: a token dictionary of 578,539 entries and a multiword dictionary of 474,800 entries. To detect words usually used with a non-medical meaning, for instance \bar", with a medical meaning (e.g. \bacilo acidorresistente"), a second analysis phase is applied with a dictionary of around 930,000 entries. In addition a mapping between SNOMED-CT and the Uni ed Medical Language (UMLS11)(1,007,705 entries) is applied. Not all these resources are needed for the BARR task, but they are already included in FreelingMed. The time problem with FreelingMed and the memory problems in the UPC approach have a direct relationship with the results in the recall column shown in Table 2.
As can be seen on both tables (Tables 1 and 2), the UNED approach outperforms by large extent every other one for both tasks and all measures. Taking into account this high di erence, combinations produce no improvement. It is 11 https://www.nlm.nih.gov/research/umls/ worth noting, however, that and combination reach the best precision for both tasks, at a cost of a extremely low recall. The NESTED type has not been treated in the entities identi cation task. 4
This paper has described our participation in the BARR task at IBEREVAL 2017 workshop, which goal is to nd acronyms and acronyms-long form relations. We have proposed ve di erent approaches, three atomic systems and two more systems, which combine on di erent ways the atomic proposals.
The UNED system clearly stands out among the presented systems. Being so clear the di erence in results with the two other atomic approaches, the combinations are not able of improving the UNED system results.
Dictionary-based approaches are language dependent while the ones based on the use of regular expressions or patterns show to be more exible.
There is, obviously, room for improvements. We plan to focus on performing combination not only as a nal process but using partial results from the other atomic sources. 5
This work has been funded by the Spanish Ministry of Science and Innovation (PROSA-MED Project: TIN2016-77820-C3, TADEEP Project: TIN2015-70214P).