This paper presents the participation of the LASTUS-TALN team in the IberLEF eHealth-KD 2019 challenge, which proposes 2 subtasks in the context of biomedical text processing in Spanish: i) the detection and classi cation of key phrases and ii) the identi cation of the semantic relationships between them. We propose an architecture based on a bidirectional long short-term memory (BiLSTM) with a conditional random eld (CRF) classi er as the last layer of the network to nd and classify the relevant key phrases. Concerning relation extraction problem, for each candidate relationship, we describe a global and local context representing the supposed relationship and the context of the candidate key phrases, respectively and divided the problem into three simpler classi cation tasks: i) decide if the entities are related, ii) identify the type of relationship and iii) obtain the correct direction. In our model, these three classi cation tasks were trained at the same time. When key phrase extraction and relation extraction were run in sequence, our system achieved the third highest F1 score in the main evaluation.
Information Extraction (IE) is the process of nding relevant entities and their
relationships in text [
Many biomedical challenges have been proposed to promote the
development of systems to extract, classify and index biomedical knowledge, such as
SemEval1, CLEF2 campaigns and others [
Previously, the eHealth-KD 2018 challenge [
Recently, in the IberLEF eHealth-KD 2019 challenge, the organizers also proposed two subtasks (see Fig. 15): i) the recognition of key phrases (subtask A, covering both, identi cation and classi cation) and ii) the detection of semantic relationships between them (subtask B). In this paper, we present our approaches and results for the participation in this challenge. From the previous challenge, we could observe that the best systems in the recognition of key phrases do not correlate with the best systems in relation extraction. Under this assumption, we propose a di erent deep learning approach for each subtask.
2 2.1
Subtask A: Identi cation and classi cation of key phrases In this section, we describe our proposal for identifying and classifying key phrases in biomedical texts. Key phrases are considered to be all the entities (single word or multiple words) that represent semantically relevant elements in a sentence.
There are four potential classes for key phrases, as described in [
For this task we propose an architecture based on a BiLSTM with a CRF
classi er as the last layer of the network. We based our implementation on the one
made available by the Ubiquitous Knowledge Processing Lab of the Technische
Universitat Darmstadt [
In order to make them compatible with the proposed architecture, we transform the format of the provided training, development and test sets into text les containing one token per line. The corresponding classes are encoded in the standard beginning-inside-outside (BIO) sequence tagging scheme.
The tokens are fed into the network as 1024-dimensional embeddings
obtained by averaging the three output layers of a deep contextualized ELMo
model [
Subtask B: Detection of semantic relations The goal of this subtask is to recognize the thirteen semantic relationships between the key phrases detected and labelled in each sentence. In addition, every semantic relation is directed, that is, the involved entities must match the correct direction.
In this subtask, we implemented a multi-task learning approach. First, we have broken down the subtask B into three simpler classi cations: i) decide if the entities are related, ii) identify the type of relationship and iii) decide the correct direction. Then, in our model, these three classi cation tasks were trained at the same time.
Before the relation extraction process, we considered as candidates all entity pairs detected in the same sentence. For each entity pair, we design a global context representing the supposed relationship and a local context representing the environment of each candidate entity.
Global and Local contexts from a relationship Following the philosophy
of [
On the other hand, we also de ned the local context of each candidate entity. The local context can provide useful information for detection of the type and direction of the relationship, as well as the presence of the relation itself. This context is also based on a sequence of words (EntityA-Context and EntityBContext ), which contains the information from the words located at the left and right of the candidate entities (with a window size of 2).
In both contexts, each sequence is represented using the concatenation of the following embeddings: tokens, PoS tags, entity types and dependencies. Model The model consists of a BiLSTM model with an attention layer on top for each concatenated embedding. The model captures the most important syntactic and semantic information from each sequence (Fore-Between, Between, Between-After, EntityA-Context and EntityB-Context) to face the three tasks: to detect if the key phrases are related, the type of relationship and its direction. In Fig. 2 a simpli ed schema of our model can be seen. In the following we explain how the model works.
First, the sentences were processed with Spacy9 to obtain the tokens, PoS tags and dependencies. From the previous subtask A, we also included the entity type information. Then, the ve sequences are generated from the tokenized sentence.
Second, the embedding layers transform each token in the sequence into a set of low-dimensions related to the token itself, the POS tag, the dependency and the entity type.
Speci cally, in the case of the tokens, an embedding layer was randomly
initialized from a uniform distribution (between -0.8 and 0.8 values and with 300
dimensions) and then, the embedding layer was updated with the word vectors
computed from the sentences in Spanish on MedlinePlus by means of fastText [
embeddings. As shown in Fig. 2, for each token of the sequence, the related embeddings are concatenated.
Next, for each sequence, a BiLSTM layer gets high-level features from its corresponding concatenated embeddings. The BiLSTM gets a word embedding sequentially, left-to-right as well as right-to-left order in parallel, producing a hidden step and keeps its hidden state through time. Therefore, it gives two hidden states as output at each step and is able to capture backwards and longrange dependencies.
A critical and apparent disadvantage of LSTM models is that they compress
all information into a xed-length vector, causing the incapability of
remembering long sequences. Attention mechanism aims to overcome the limitation
of xed-length vector keeping relevant information from long sequences.
Attention techniques have been recently demonstrated success in multiple areas of the
NLP such as question answering, machine translations, speech recognition and
relation extraction [
Then, the nal relation-level feature vector produced by the previous BiLSTM layers feed the following dense layer, which directs its output to three parallel fully-connected layers to classify each task. Note that the three nal output layers are connected in cascade, that is, the output of the rst classi cation (are these entities related?) also feeds the second classi cation task (type of semantic relationship?), and the last one (direction of the relationship?) is feed by the outputs from the previous dense layer, the rst and the second classi cation task.
Finally, we consider that a relationship has been detected when our model predicts a positive value in the three classi cation tasks. Team F-Score Precision Recall LASTUS-TALN 0.8167 0.7997 0.8344 Highest Score 0.8203 0.8073 0.8336 Average Score 0.7749 0.7746 0.7774 Baseline 0.5466 0.5129 0.5851 The organizers proposed a main evaluation scenario (Scenario 1) where subtasks A and B are performed in sequence. Additionally, two optional scenarios were considered in order to evaluate each subtask independently of the other (Scenario 2 for subtask A and Scenario 3 for subtask B).
Results of our system for each scenario are given in Tables 1, 2 and 3. In addition to our results, we include the highest and average scores obtained for all the participants in each scenario. Please note that, as there was a bug in our submitted implementation for subtask B, which was subsequently xed, Tables 1, 2 and 3 show the o cial results achieved in the challenge as well as the xed results, which we comment bellow.
For subtask A, our system achieved one of the best results in the challenge (see Table 2). In contrast, for subtask B, our system obtained one of the lowest scores (see Table 3). When the tasks are performed sequentially (see Table 1) the errors in subtask B are mitigated by the good performance obtained for subtask A, leaving our system in the third position of the challenge. 4
In this paper, we presented the participation of the LASTUS-TALN team in the
IberLEF eHealth-KD 2019 challenge, which proposed 2 sub-tasks, the detection
and classi cation of key phrases and the identi cation of the semantic
relationships between them. In the rst scenario, we proposed a BiLSTM network for
sequence tagging, based on the architecture made available by the Ubiquitous
Knowledge Processing Lab of the Technische Universitat Darmstadt [
On the other hand, we followed a daring philosophy to represent relationships in multiple context. Although our results for task 2 were not as expected, we think that this representation has a lot of potential. For that reason, our future work will focus on the study of this representation and how it behaves in neural network. In this sense, we want to achieve a performance close to the state-ofthe-art in this challenge.
Funding: This work is partly supported by the Spanish Government under the Mar a de Maeztu Units of Excellence Programme (MDM-2015-0502).