While clinical text NLP systems have become very e ective in recognizing named entities in clinical text and mapping them to standardized terminologies in the normalization process, there remains a gap in the ability of extractors to combine entities together into a complete semantic representation of medical concepts that contain multiple attributes each of which has its own set of allowed named entities or values. Furthermore, additional domain knowledge may be required to determine the semantics of particular tokens in the text that take on special meanings in relation to this concept. This research proposes an approach that provides ontological mappings of the surface forms of medical concepts that are of the UMLS semantic class signs/symptoms. The mappings are used to extract and encode the constituent set of named entities into interoperable semantic structures that can be linked to other structured and unstructured data for reuse in research and analysis.
Medical natural language processing (NLP) systems have become very e ective
in recognizing and normalizing named entities in clinical text by mapping them
to standardized biomedical terminologies such as those contained in the Uni ed
Medical Language System (UMLS) [
Additionally, some attributes are idiosyncratic for a particular medical
concept and require speci c domain knowledge. For example, pain severity can be
noted in the chart using di erent scales such as the Wong-Baker Faces scale
[
This research proposes an ontology-based data access approach to retrieving semantic representations of composite medical concepts of the type sign or symptom. This lightweight, highly scalable approach allows for clinical data to be integrated with other structured data for use in clinical decision support or to be reused in biomedical research for cohort identi cation and analysis. The research described in this paper is being performed in conjunction with the University of Maryland Medical School and the Baltimore Veterans Administration (VA) Hospital Emergency and Radiology Departments. This research came as a result of an expressed need by the physicians at the VA to enhance care by integrating semantic search within a patient's chart into their electronic health system. These physicians were particularly interested in monitoring a patient's pain over time.
This research focused on extracting and encoding the medical concept of pain both as a proof of concept and because of the physicians' strong interest in it. Pain is an especially di cult target because of its prevalence in human experience. This results in a very wide variety of ways in which the concept of pain and its severity and quality may be expressed. In this ongoing work, this paper will detail the approach as it relates to pain and its severity. 2
Several medical NLP systems have been developed which are e ective in
normalizing and annotating medical concepts in clinical and biomedical text such
as cTAKES (clinical Text Analysis and Knowledge Extraction System) [
Other systems such as Medical Language Extraction and Encoding System
(MedLEE) [
More recently, the Strategic Health IT Advanced Research Projects Research
Focus Area 4 (SHARP4) program aims to develop open-source tools for
largescale health record data sharing. One of the core initiatives is the development
of the Clinical Element Model (CEM) [
The corpus consists of segments of ten deidenti ed patient charts that were
downloaded from the VA's VistA [
The general approach pursued in this research is that of ontology-based data access over unstructured data. Two ontologies and one analysis engine are used for each medical concept and for each attribute that is a constituent part of that concept. The rst ontology assists in mapping the surface forms of the medical concepts in the text to their normalized representation. The second ontology de nes semantic representation of the medical concept. These ontologies are implemented separately so that inference and reasoning over the knowledge base of facts is not encumbered by knowledge that is only needed for extraction. The semantic representation will use the CEM/OWL representation of the sign and symptom class as described above. The analysis engine uses the knowledge in the mapping ontologies to extract and encode them into their semantic representations. Separating the mapping and semantic representation from the analysis engine also allows alternative analysis engines to be used. These triple-sets form composable building blocks. For instance, onset and duration can be reused as attributes of many other medical concepts that are of the type sign and symptom.
The upper level mapping ontology contains all the knowledge needed to extract the main medical concept of interest. It also imports the subontologies for its constituent attributes such as body location and severity. The sequence of extractions is performed based on the subontologies. When order is important, (e.g., severity depends on the body location extractor), the order is speci ed in the top level mapping. When order is not important, attribute extraction may be parallelized to increase throughput.
For medical concepts and attributes in which a particular vocabulary has been assembled, such as for pain and severity, these terms are stored in the mapping ontology. Each term is a subclass of a Term class in which the base form is stored along with a regular expression that matches its surface lexical forms. Other domain knowledge needed for the extraction of the term may also be asserted in the class. Terms may also be subclasses of other terms. For example, in the severity mapping, the term Minor may be a subclass of the term Mild for normalization purposes.
For mappings that require external sources, the mapping ontology contains
references to resources such as terminologies, lexicons, and other ontologies. For
example, the body location mapping ontology contains references to
Foundational Model of Anatomy (FMA) [
Once the extraction is complete, the pertinent information is encoded as an instance in the ontology that contains the semantic representation of the medical concept. This includes the reference to the document level information, normalized concepts identi ers, the original text found, and the line number and text span in which it was found in the note. 3.3
For many medical concepts of the class signs and symptoms, severity is expressed generally as mild, moderate, and severe. However, it may also be expressed with a large number of synonyms that imply some level of pain such as minimal or torturing. Many of these terms are found in one or more medical terminology systems. Others are only found in general English thesauri. Severity terms were mined from these sources as well as the development data, pain questionnaires, online health sites, and research papers. The terms were then added to the mapping ontologies. With the help of physicians, these terms were related as subclasses to the normalized terms of mild, mild to moderate, moderate, moderate to severe, and severe which have been adopted by SHARP. As pain may also be expressed as a value on a numeric scale, these values were also normalized.
Listed below are various ways that pain severity was expressed in this corpus.
1. \SCALE:7", \Pain:4"
2. \Patient reports pain during shift: No"
3. \pain was tolerable", \minor abdominal soreness", \denies any pain"
4. \s: pain: 8.5/10 b hands"
Algorithm. The pain severity analysis engine modi es and extends the
graphbased ConText [
The algorithm processes one textual unit, such as a sentence, at a time. The scope of target nodes is the entire textual unit. The scope of modi er nodes begins at the start span of the modi er term in the text and ends at either the beginning or end of the textual unit depending on the direction of its rule. Target nodes are connected to modi er nodes whose scope falls within their own.
Because this algorithm does not rely on syntactic parsing, it was very e ective on the type of non-canonical text entered by the clinician at the point of care that is prevalent in this corpus such as: \64 yo AAM with h/o stage IV RLL adenocarcinoma T2N0M1 s/p treatment with carboplatin/taxol"
be very di cult with this type of text, scoping was changed to treat the newline characters as the basic unit of scope. Scoping modi cations were also made for other types of punctuation such as semicolons and periods.
the database to identify the di erent expressions of pain and its severity. Seventy modi er rules were also added . Twelve were of the type DEFINITE NEGATED EXISTENCE to accommodate the idiosyncrasies of this corpus such as \Pain during shift: No". Forty-eight were of the new type PAIN SEVERITY to recognize both numeric and lexical terms indicating severity (\9/10",\not too bad"). Ten were of the type PROBABLE EXISTENCE (\c/o",\states"). 4 4.1
The reference standard was created by three physicians. The rst physician provided oversight concerning what aspects of the data would be extracted which was used to develop the annotation guidelines. Two other physicians independently annotated each line of text in the test set for two variables, i.e, Existence and Literals. Existence was labeled with A rmed, Probable, Negated, or N/A. Literals were labeled with the speci c severity quanti er found in the text - numeric or lexical. If an expression crossed a line boundary, the annotation was made on the rst of the two lines. The rst physician annotated text in cases where the two annotators did not agree. This annotation was used to break the tie for the nal gold standard.
A combination of the Existence and Literals annotations were used to create the nal Severity variable against which the system was tested. If a line contained a Literal value, that value was used for Severity. Otherwise, if Existence was labeled A rmed or Probable, Severity was labeled Unknown. If Existence was labeled Negated, Severity was labeled 0.
For Existence, Ao = 0:86(290=337); Ae = 0:468, and Fleiss's = 0:738. For Literals, Ao = 0:79(79=100); Ae = 0:184, and Fleiss's = 0:743. 5
Results were evaluated using the vertical metrics for measuring system
performance on individual elds as outlined in [
#V alues returned by the system recall = #Correctly returned values by system #V alues in gold standard (1)
This resulted in a positive precision of 90.31% (233/258), a positive recall of 88.26% (264/289), and a positive F1-score of 89.27%.
For comparison, the original ConText algorithm reported very good results
for negated ndings in a corpus of six di erent note types (P:97%, R:97%) [
In these results, recall su ers due to not recognizing lexical severity values (\mildly"), misspellings (\dnied"), and missing whitespace (\Chest Pain{denies"). However, almost one-third of false negatives (11/31) were the result of pain mentions that cross line boundaries (\no c/o [new line] pain"). For text that was characterized by non-canonical use of grammar, attempts to ignore line breaks and segment into sentences was not successful and resulted in lower accuracy. However, for the more canonical text found in specialist reports (radiology, surgery), this would be bene cial. Plans for future work include the use of machine learning techniques to train a classi er to recognize these di erent types of text.
Another source of error in recall was when a pain score was associated with body locations instead of to a target term, for example, \7.5 both hands." When work on the body locations extractor is complete and can identify these terms in the text, the severity extractor should be able to handle these cases as well.
An additional di culty is that there may be more than one pain mention within the same line/sentence, for example, \new onset headaches, no c/o chest pain". Likewise, several body locations may share the same negation modi er, such as, \pt denies any abdominal pain, headaches," The algorithm will need to be modi ed to handle these as separate pain mentions.
An interesting issue came up regarding which terms imply pain. In the narrative note, terms such as pressure, discomfort, and sensation are treated di erently from pain, for example, \feels pressure, no pain." In this case the algorithm classi ed it as a negated pain mention, but the physician annotator classi ed it as Probable pain with Unknown severity. In addition, the term tender was not regarded by the physician as an indicator of pain if it appeared in a physical exam section of a note and was the result of a palpation, for example, \Abd: tender, nondistended". However, the physician did classify it as pain if it appeared in a triage note, for example, \redness w/ warmth and tenderness." The semantics will need to be extended to recognize the broader context such as the type of note and sections within a note. This type of domain knowledge can be asserted in the mapping ontology used to encode the entity into its full semantic representation.
This ongoing research investigates an approach that allows clinicians to perform semantic search through a patient record to explore diagnostic hypotheses. It extracts and encodes the medical concept of pain into a semantic whole that can be stored in a knowledge base and used with reasoning and inference engines for clinical decision support and for reuse in analysis. In addition, it operates over all note types, including the very di cult non-canonical text that typi es narrative notes that are entered by clinicians at the point of care. Future work will undertake to extend it to other concepts of the type signs and symptoms.