Medical records contain relevant information about patients, which can be bene cial to improving healthcare and research in the clinical domain. Due to this, there is a growing interest in developing automatic methods to extract and exploit the information from medical records. However, medical records also contain protected health information about patients. To protect the con dentiality and privacy of patients, this sensitive information should be removed prior to any processing of these documents. In this paper, we describe an architecture for the detection and identi cation of protected health information from medical records. The architecture is composed of two bidirectional Long ShortTerm Memory layers and a nal layer based on Conditional Random Fields. Our system participated in the Meddocan shared task, obtaining a micro-F1 of 93.22%.
In recent years, the number of electronic medical records (EHRs) has been increasing massively. These registries are very useful resources to perform studies focused on detection/prevention of diseases and medical decision making, among others. However, health records contain protected health information (PHI). For instance, information about family history, treatment tracking, and data that may help to identify a given patient (for example, patient's name, address, telephone number, zip code, etc). Due to this protected information, medical records cannot be shared without a previous de-identi cation process. This process consists of detecting and subsequently replacing or removing all protected information from the records.
The interest in addressing de-identi cation problems motivated the proposal
of two tasks, the 2006 [
The identi cation of PHI inside medical documents can be addressed as a
Named Entity Recognition (NER) problem. This problem has been widely
studied and the approaches normally used can be divided into several main
categories: dictionaries and rule-based systems, machine learning, deep learning,
and hybrid systems. Dictionary-based methods are limited by the size of the
dictionaries themselves, in addition to the constant growth of vocabulary and
spelling errors. Rule-based approaches usually provide high precision, however,
they do not usually contemplate all existing cases as a result of the complexity
of the language. Furthermore, rule-based and machine learning methods require
a previous generation of syntactic and semantic features, as well as
domainspeci c information. Approaches based on deep learning methods automatically
learn relevant patterns, allowing a certain grade of independence of language
and domain. Moreover, these approaches have been shown to achieve better
results than the best hybrid systems in i2b2 tasks. The system described in [
Considering the above, in this paper, we propose the use of an adaptation
of the NeuroNer tool [
The training and development sets provided in the MEDDOCAN-2019 task are composed of 500 and 250 clinical cases respectively and contain 22 di erent types of PHI entities listed in Table 5. In both sets, the representation of the di erent types of entity is proportional and unbalanced.
Protected health information recognition by BiLSTM-CRF
The clinical cases are initially provided in BRAT format1, a stando format where the di erent annotations are stored separately from the original text in a similar way to the BioNLP Shared Task stando format2. 3 3.1
We pre-process the text of the clinical cases taking into account di erent steps. First, sentences are split using the Spacy (Spacy.io), an open-source library that provides support for texts in several languages, including Spanish. Due to the nature of the language used in this type of text, we have de ned a set of rules to avoid detecting dots corresponding to acronyms or abbreviations as sentence separators.
Afterward, the resulting sentences are tokenized by using Spacy. However, some tokens meet the following pattern " eld:value". For instance, the token "cp:28007", which refers to a postal code, should be split into the corresponding eld ("cp") and its value ("28007"). Due to this, we have included a set of rules to correctly split this kind of tokens.
Finally, the text and its annotations are transformed into the CoNLL-2003
format3 using the BIOES schema [
As we can appreciate in section 1, approaches composed of Bidirectional LSTMs
layers in conjunction with CRF, provide good results in NER tasks [
The rst layer aims to generate vector representations of the tokens that
conform the input sequences. The direct representation of token to vector (word
embedding) can be pre-trained or can be learned in conjunction with the rest
of the model by adjusting its weights. Pre-trained models can be obtained from
a large amount of unlabeled data with methods such as word2vec or GloVe [
The character embedding sequence of each token is passed as input to a BiLSTM to obtain character-based word embedding as output. Finally, the representation of word embeddings and character-based word embedding are concatenated for each token, which will be the input for the second BiLSTM layer. This BiLSTM layer aims to obtain the sequence of probabilities for each token to pertain to a given label using the BIOES coding. The label for each token will be the one with the highest probability.
The last layer consists of a conditional random elds layer. This layer receives as input the sequence of probabilities of the previous layer in order to improve predictions. This is due to the ability of the layer to take into account the dependencies between the di erent labels. The output of this layer provides the most probable sequence of labels.
The parameters of the embedding and hyperparameters of our model used for the MEDDOCAN-2019 task are listed below: { Word Embeddings: randomly initialized and adjusted during training.
The dimension of the vectors is 100. { Character Embeddings dimension: randomly initialized and adjusted during training. The dimension of the vectors is 25. { First BiLSTM hidden state dimension: 25 for the forward and backward layers { Second BiLSTM hidden state dimension: 100 for the forward and backward layers { Optimizer: Stochastic gradient descent (SGD), learning rate: 0.01 { Dropout: 0.5 { Number of Epochs: 100 4
The MEDDOCAN-2019 evaluation process consists of two di erent scenarios. The rst scenario consists of detecting exactly the location in the text of each PHI, as well as the type of entity. The second scenario consists of identifying
Protected health information recognition by BiLSTM-CRF sensitive data in order to be replaced, regardless of the type of entity. In the last scenario, two di erent types of evaluations are performed: (1) Strict evaluation of the spans of PHI that belong to sensitive phrases. (2) Merge evaluation where the spans of PHI connected by non-alphanumerical characters are merged. To evaluate both scenarios, the metrics proposed by the organizers are micro-averaged precision, recall, and F1-score.
To evaluate the trained models, as well as their hyperparameters, we performed a set of experiments with the development dataset provided by the MEDDOCAN-2019 organizers. We used grid search to adjust the word embeddings dimension, the number of units in the BiLSTM hidden layer, the optimizer and the learning rate.
We can observe in Tables 2 and 3 that our best results on the overall
development set do not present signi cant di erences. The run0 model was trained
using the hyperparameters mentioned in section 3.2. The run1 model was trained
using ADAM [
Considering that both the models run0 and run1 achieved the best results in the two subtasks, these models were used to process the test set provided by MEDDOCAN-2019 in scenarios 1 and 2. The results obtained in both tasks can be seen in Table 4. Moreover, we can verify that the run0 model is the one that provides the best results in all scenarios (F1 of 93.22% in sub-task 1, F1 of 94.26% in sub-task 2 for strict evaluation and F1 of 95.77% for merged evaluation).
Once the run 0 model has been selected as our best model, it is interesting to perform a more detailed study for each of the PHI due to the unbalance of the problem. As we can see in Table 5, not all types of entities are classi ed so easily. For example, entities such as ID EMPLEO PERSONAL SANITARIO and OTROS SUJETO ASISTENCIA are never correctly classi ed. This may be due to their low representation in the training set, as well as the confusion it may cause with other similar types of entities such as FAMILIARES SUJETO ASISTENCIA. On the other hand, the types of entity with the best results are those with the highest representation in the training set or those with a speci c structure such as ID ASEGURAMIENTO and ID CONTACTO ASISTENCIAL. 5
Medical records are sources of a wide range of studies to detect and prevent diseases, as well as to provide information for medical decision making. The growing volume of these types of records, in addition to the studies associated with them, increases the importance of de-identi cation. This type of process
Protected health information recognition by BiLSTM-CRF has the goal of eliminating or replacing sensitive data to allow access to medical information without compromising the identi cation of the patients involved.
Most previous e orts in the task of anonymization medical records have been focused mostly on texts written in English. Meddocan is the rst shared task devoted to the anonymization of medical records in Spanish. One of the major challenges of this shared task is that there are a large number of sensitive data categories (22 di erent types of entities). In addition, these data are often unbalanced in the text. This results in di culties to classify them correctly.
In this paper, we describe our participating system in this task. It exploits the NeuroNer tool, a tool based on deep learning with bi-directional LSTM and CRF layers for the task of NER. In spite of the challenges above described, our system obtains a micro-F1 of 93.22% on the test set.
For future works, we plan to explore other deep learning architectures as
well as exploiting pre-trained word embedding models, as well as other types of
embeddings such as sense embeddings [
This work was supported by the Research Program of the Ministry of Economy and Competitiveness - Government of Spain (DeepEMR project TIN2017-87548C2-1-R).