In this paper we present a Natural Language Processing (NLP) pipeline to automatically extract medical concepts from a free text written in a language other than English. To do so, we use common NLP techniques and the metathesaurus of Unified Medical Language System (UMLS). Specifically, our goal is to automatically extract ontological concepts representing which part of the human body is injured and what is the nature of the injury, given an Italian textual description of a work accident. We start by partitioning the text into tokens and assigning to each token its part-of-speech, and then use an appropriate tool to extract relevant concepts to be searched within UMLS. We tested our system on a public large repository containing textual descriptions of work accidents produced by INAIL. Experimental results confirm that our system is able to correctly extract relevant medical concepts from texts written in Italian.
The term Electronic Health Records (EHRs) describes the concept of a comprehensive,
crossinstitutional, and longitudinal collection of healthcare data, trying to group the entire clinical
life of a patient [
In this work we focus on the natural-language texts included in EHRs, since unstructured data analytics is one of the most challenging task of EHRs automated analysis. Our main contributions are the development of a system capable of automatically extracting medical ontological concepts from an Italian natural language text, and the experimental evaluation of our system using a real-world dataset regarding work injuries.
The paper is organized as follows. Section 1 gives an overview of the problem, Section 2 reviews the state of the art regarding medical concept extraction, Section 3 describes our methodology, Section 4 presents the experiments and, finally, Section 5 concludes the paper.
Concept extraction from natural language texts related to clinical information consists of three
phases [
We now briefly describe one of the most complete medical ontology system (UMLS) and two state-of-the-art frameworks for clinical concept extraction: cTAKES and MetaMap.
The Unified Medical Language System (UMLS) [
UMLS is composed by three modules: Metathesaurus, Semantic Network, and Specialist Lexicon. We now provide a brief explanation of each of these modules.
The Metathesaurus of UMLS includes over one million biomedical concepts and five million concept names, enclosing many vocabularies such as ICD-10, SNOMED CT, MeSH, and more. The Metathesaurus is structured to facilitate the identification of synonyms between concepts, also in diferent languages ensured by leveraging hierarchical concept identifiers, in turn linked to: • Concepts (CUI): identifying the meaning to be expressed, it never changes over time, no matter the updates in the vocabularies. • Strings (SUI): each string representing a concept is assigned with a permanent string identifier. Any character variation (e.g. case sensitivity, punctuation, etc.) will result in a diferent SUI, for each language. A SUI can in principle be linked to more than one CUI. • Atoms (AUI): being building blocks of the Metathesaurus, atoms represent specific entries in the vocabularies included in UMLS. An AUI is linked to one and only one CUI. • Lexical terms (LUI): a lexical term comprises diferent strings (i.e. SUI) that are lexical variants or minor variants. This layer is often used to reduce the computational complexity of exploration and for a more efective concept lookup. It is currently available for all the English strings, and only partially for other languages.
The Semantic Network provides a consistent categorization of all concepts represented in the Metathesaurus along with a set of useful relationships between these concepts. The network contains 133 semantic types and 54 relationships. Each concept in the Metathesaurus is assigned one or more semantic types, which are linked to one another through semantic relationships. The major semantic types are organisms, anatomical structures, biologic function, chemical, events, physical objects, etc.
The possible relationships between semantic types range from simple ⟨is-a⟩ hierarchies to complex associations, such as ⟨physically related to⟩, ⟨spatially related to⟩, ⟨co-occurs with⟩. Relationships can be derived from associations already present in the vocabularies (intrasource relationships) or they can connect concepts from diferent vocabularies ( inter-source relationships), including not only synonyms. Inter-source relationships enhance the integration of all the vocabularies present in UMLS and enable an easy exploration of the resulting ontology. A subset of the Semantic Network with ⟨is-a⟩ relationships is shown in Figure 2.
Specialist Lexicon is a module of UMLS that addresses the high degree of variability in natural language words, allowing the abstraction of lexical variants. It covers general English lexicon and many biomedical terms, including syntactic, morphological, and orthographic information. Since only English words are covered by this module, we adopted a diferent approach for Italian Natural Language Processing, which we will describe in Section 3.
Apache clinical Text Analysis and Knowledge Extraction System [
MetaMap was developed by the National Library of Medicine (NLM) with the goal of mapping
biomedical text to the UMLS Metathesaurus [
A comparison between cTAKES and MetaMap has already been investigated in [
The main disadvantage of both cTAKES and MetaMap, with respect to our study, is that they are strongly English-centric, since they both rely on UMLS Specialist Lexicon which, as we already described in Section 2.1.3, fully covers only the English language.
We now present the methodology of our system. The final goal is to automatically extract ontological concepts representing which part of the human body is injured and what is the nature of the injury, given an Italian textual description of a work accident. Figure 3 displays the three phases of our workflow: Part-of-Speech (POS) Tagging, Keyphrase Extraction and Dictionary Lookup.
The first phase receives as input a textual description representing the dynamic of an accident
and gives as output a preprocessed and enriched representation of the input text. Specifically,
Tint takes as input a raw text in Italian and performs a series of natural language processing
operations. Tint (The Italian NLP Tool) [
Example 1: Given the following accident description: “Erano in corso attività di produzione di acciaio. Mentre un agganciamento del nastro trasportatore vibrovaglio alla motopala, a causa di una manovra pericolosa rimaneva con le braccia in contrasto tra le due macchine decedendo per contusione al fegato”, the pipeline produces the tagged text visible in Table 1.
The second phase receives as input the tagged text with lemmas and POS, and returns as
output a new series of keyphrases ordered by importance and frequency. This step uses a tool
called Keyphrase Digger (KD) [
Example 2: After the second phase the pipeline extracted the following concepts: “produzione di acciaio”, “nastro trasportatore vibro”, “contusione al fegato”, “vaglio alla motopala” and “manovra pericolosa”.
As can be seen from Example 2, not all the concepts extracted from the accident description contain information regarding the part of the body and the nature of the incident. The third and last phase, exploiting the Rest API provided by the UMLS, allows the system to query various databases and to discard all the concepts that do not relate to the medical field. This phase can be divided into two distinct sub-phases: • Concept lookup: we create a query that queries UMLS to get the medical concept. We use as input every possible combination of the keyphrases obtained in the previous step. • Semantic type lookup: after obtaining the medical concept, we check if it belongs to one of these four semantic types, which represent nature and location of an injury.
Example 3: After the third phase the only keyphrase not discarded is “contusione al fegato”.
We tested our system on the accident descriptions contained in the InforMo dataset [
For each text, our system must extract two concepts that will form a pair constituting the nature and location of an injury. What is extracted is compared with the golden truth to assess the accuracy of the framework. What we want to achieve is an exact extraction of the nature-location pair directly from the textual description of the accident.
After analyzing each textual description, we have evidence to say that most of the times when a nature-location pair is present in the text, it is in the same period, so we analyze each period. If a couple is present in a period of the text we keep it, otherwise we delete the couple. This allowed to greatly reduce the extracted concepts while keeping the performance unchanged.
To evaluate the performance of our pipeline we used recall, precision and F1-score, whose definitions are reported reported here:
Recall :=
+
Precision :=
+
Precision · Recall F1-Score := 2 · Precision + Recall
TP represents a correctly guessed nature-location pair, the FP instead are all those incorrect but still extracted, FN are those that are mistakenly not recognized as a match.
In the UMLS query we can specify a parameter called “searchType" which can take two diferent values: “words" (by default) or “exact". With the first, a similarity search is carried out, resulting in the list of concepts most similar to the one given in input, ordered by decreasing similarity. With the second, on the other hand, a result is obtained only if the input word really exists in the database. We tested the system checking both of these parameters so as to understand which is the best one. In the Table 2 we list the results in the two cases. The results show that the “exact" case is better than the second. A deeper analysis highlights that this it is due to the fact that in the second case many more concepts are extracted, most of which are quite diferent from the original one.
In this paper we presented a methodology for extracting medical concepts from accident descriptions written in natural language, specifically tailored for the Italian language. The system, still being in a preliminary phase, sufers from some limitations: (i) there is a strong dependence on UMLS and its provided APIs, this often makes the system pretty slow in its computation (ii) the experimental campaign carried out is pretty limited, this may cause problems in the applicability of the framework to other input sources. This work has been funded by INAIL within the BRiC 2018, ID09 framework, project RECKON.