Automating the extraction of essential patient information from clinical texts, such as admission notes, can significantly enhance the entry of this data into Electronic Health Records (EHR), thereby enhancing workflow eficiency and supporting improved patient care and healthcare management. To address this issue, we introduce a Natural Language Processing (NLP) pipeline designed to (i) automatically extract patient data via Named Entity Recognition (NER), (ii) normalize the extracted data to correspond to codes in oficial medical ontologies, and (iii) coerce the data into EHR format using Health Level 7's (HL7) Fast Healthcare Interoperability Resources (FHIR) standard. By adhering to these widely used standardized formats, the pipeline output can be immediately integrated into the Hospital Information System (HIS). To achieve this, we propose a newly labeled dataset comprising 255 notes from unlabelled datasets published by the Text Retrieval Conference's (TREC) Clinical Trials tracks. Finally, we utilize SapBERT for the normalization of extracted entities and employ the FHIR standard as a basis to generate Electronic Health Records (EHRs).
Representations from Transformers (BERT) [
For the first part of our pipeline, we largely rely on microsoft/mdeberta-v3-base [
For coercing the output into an EHR, we have decided to use Fast Healthcare Interoperability Resources (FHIR) in the third step of the pipeline. FHIR is a standard for exchanging healthcare information electronically, designed to enable interoperability between diferent healthcare systems. Since it is designed to be extensible, it allows developers to build custom applications and extensions without jeopardizing compatibility - this is a big factor on why we use FHIR as the output format of our pipeline.
In this section, we introduce the initial phase of our clinical text processing pipeline, focusing on the extraction of crucial patient-related information from admission notes. We look into specific objectives and evaluation metrics, and we introduce and explore a newly labeled dataset. We look into the training and fine-tuning of various BERT models, along with strategies for enhancing their performance.
The primary dataset originates from the TREC CT/CDS topics, publicly accessible on the track’s oficial
website1. Each topic has a similar structure, including several diagnoses in free text format. The topics
represent admission notes containing the most important patient details which a doctor takes as soon
as a person is admitted to a hospital. This includes personal information and demographics, such as
gender and age, but also the current medical conditions, symptoms, medications/treatments and medical
procedures. The dataset makes a total of 255 entries (topics). This includes:
• TREC CDS 2016 [
For simplification purposes, we have decided to focus on four diferent entities, encompassing the most important information which has to be extracted from admission notes.
• Medical Conditions
Medical conditions describe long-term conditions, such as diabetes mellitus or COVID-19. • Symptoms
In contrast to medical conditions, these describe mostly short-term conditions, which may be indicators of medical conditions. E.g. fever, a symptom, is an indicator for COVID-19, a medical condition. • Medication/Treatment
This could either describe medicine (e.g. Ritalin) or treatment (e.g. rehab). • Medical Procedure
This includes both invasive and non-invasive procedures, such as tracheostomy or MR. The labelling of the dataset has been done via the open-source tool doccano [17] by the author of this paper. See Table 1 for a summary of annotated data by entity type, showcasing how imbalanced the entity to non-entity ratio is. It is important to mention that this specific dataset has not been reviewed by domain experts. There were several issues while labelling the data. The term medical condition is not entirely clear and subject to interpretation. For instance, we can observe the relationship between medical condition and symptom. E.g., a fever is not a medical condition - it is a response to medical condition or disease. The same goes for dysuria, being the subsequent response to e.g. UTIs (Urinary Tract Infections), a collection of various medical conditions. On the other hand the question arises whether injuries can be seen as medical conditions. In fact, injuries, such as a broken arm, are not considered medical conditions - injuries themselves are their own category in the medical language, which are, however, not included in this analysis.
For evaluating the models, we used entity-level evaluation metrics [18], consistent with our previous
submission for MultiCardioNER [
As can be seen in section 3.1.2, there exist great imbalances in the relative ratio between entities and non-entities. In order to tackle this problem and increase model accuracy, we have decided to even these modalities out via data augmentation. In detail, we shufle the sentences and their respective entities around in random order and thus generate new model input, essentially doubling the amount of training data. This augmentation has only been performed on the train set, while the validation and test set were left unchanged. For sentence detection, we have used spaCy [21]. In general, this resulted in overall increased metric values, as can be seen in Table 2; bear in mind that they originate from already fine-tuned models.
In this section, we address both the second and third steps of our pipeline. We introduce the standard medical ontologies utilized in our work, and we also dive into the technology used for normalizing the extracted information, specifically focusing on the application of SapBERT. Following this, we discuss how we used the HL7 FHIR standard to integrate all extracted and normalized data into an EHR system. For entitiy types Medical Condition, Medication/Treatment and Symptom we use the ICD-10 codes taken directly from the website for Centers for Medicare & Medicaid Services2, and for Medication we use NDC codes. These were taken from the FDA’s oficial website openFDA 3, but had to be thoroughly preprocessed for use. We used both the proprietary and non-proprietary name for matching the code. Furthermore, since each NDC code includes packaging information, which we do not extract from the text, we have arbitrarily selected one code to represent the medicine. As a result, the packaging details associated with this code may not be accurate.
Entity Embeddings
ICD/NDC
Embeddings
Nearest Neighbor (k=1) Consine Similarity Threshold
SapBERT Extracted Entities
ICD/NDC Codes f e a tr u e s p a c e r a w d a t a
The integration with SapBERT is required for medical entity normalization. In order to standardize the extracted entities, we need to connect each of them with their respective ICD-10/NDC code. In 2https://www.cms.gov/medicare/coding-billing/icd-10-codes 3https://open.fda.gov/data/ndc/ detail, we leverage SapBERT to create embeddings for ICD-10 and NDC. The core steps of the workflow (see Figure 2) include generating embeddings for these codes, performing nearest neighbor search, and determining the cosine similarity between embeddings to find the closest matches based on a pre-defined threshold. Based on experience from initial experiments, we chose 0.4 for ICD-10 codes and 0.3 for NDC codes.
The input of this part is (i) the extracted text of the NER model, (ii) the normalized entity and (iii) the corresponding ICD-10/NDC code. These triples are then grouped into the fitting FHIR resource, which represent specific types of clinical and administrative information in the FHIR standard. The HL7 organization ofers a FHIR Resource Guide 4, with which it was quite simple to find and use the appropriate resources. The FHIR resource templates for each separate entity type and an example for a FHIR Resource Bundle as a final EHR can be found in the GitHub repository. It is interesting to note that the resources templates for Medical Condition and Symptom are the same, except for a note being used to highlight the diference. This once more emphasizes the content-related overlap of these entity types as described in section 3.1.2.
One of the primary limitations of this work lies in the lack of a quantitative evaluation of the mapping methodology introduced in Section 4.1. While the approach to map extracted entities to standard codes shows promise, we do not provide a formal assessment of its performance. Future iterations of this study should aim to address this gap by introducing an appropriate evaluation framework, allowing for a stronger argument regarding the efectiveness of the mapping mechanism and its applicability in real-world scenarios. Furthermore, the dataset used in this study, consisting of approximately 32943 wordpieces, poses potential challenges for the generalizability of the findings. A dataset of this size, while suficient for a proof-of-concept, may not capture the full complexity and variability present in larger, real-world datasets. Moreover, the dataset lacks expert intervention during the labeling process, which introduces the possibility of inaccuracies in entity extraction. In future work, incorporating expert validation for at least a subset of the data would enhance the quality and accuracy of the annotations, providing a more robust foundation for the entity extraction and mapping methods.
We have established an NLP pipeline for processing free-text admission notes into EHR. In Section 3, we look into the problem of medical NER. We select the appropriate architecture and define metrics for demonstrative evaluation. We describe the data mining process, explore the data and justify our data labelling processes. Finally, we train models through various strategies and assess their performance. In Section 4, we gave insights into medical classification lists such as ICD-10 and NDC. We further showcase how we take the output of Section 3 to normalize the extracted entities via SapBERT. Finally, we show how the FHIR standard aids us in generating a standardized EHR. Furthermore, we have created a web interface to showcase all three steps of the pipeline: (i) the extracted entities inside the admission note, (ii) the normalized entities, including the extracted text, the normalized text and ICD-10/NDC code, and (iii) the automatically generated FHIR Resource Bundle representing a standardized EHR. Any code can be found in the GitHub repositories Padraig20/Disease-Detection-NLP and Padraig20/EHR-Generator for medical NER and the EHR-Generator including the SapBERT workflow as well as the Web-Interface, respectively. 4https://www.hl7.org/fhir/resourceguide.html in: Proceedings of the Text REtrieval Conference (TREC), National Institute of Standards and Technology (NIST), Gaithersburg, MD, USA, 2023. [17] H. Nakayama, T. Kubo, J. Kamura, Y. Taniguchi, X. Liang, doccano: Text annotation tool for human, 2018. URL: https://github.com/doccano/doccano, software available from https://github.com/doccano/doccano. [18] D. S. Batista, Named-entity evaluation metrics based on entity-level, 2018. URL: https://www.
davidsbatista.net/blog/2018/05/09/Named_Entity_Evaluation/, accessed: 2024-05-21. [19] J. Devlin, M.-W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, in: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), 2019, pp. 4171–4186. [20] Á. Alonso Casero, Named entity recognition and normalization in biomedical literature: a practical case in SARS-CoV-2 literature, 2021. URL: https://oa.upm.es/67933/, unpublished. [21] Explosion-AI, spaCy: Industrial-strength Natural Language Processing in Python, https://spacy.io/ usage/linguistic-features#sbd, 2023. URL: https://spacy.io/, version 3.0.