The TURBO Medication Mapper (TMM) identifies terms from RxNorm that best represent a list of medication strings, like one would find in a Clinical Data Warehouse (CDW). TMM has several differentiating characteristics, compared to other tools: the machine learning component does not require a human-curated gold standard for training; normalizations are applied to source-specific language in the strings (instead of just excluding them); the confidence of each mapping is represented as a relationship to the absolute truth, along with a 0.0-1.0 score; the results, along with supporting knowledge, are saved into an RDF graph and a Solr document database is generated. Queries for drug classes like “statins” are based on OBO foundry ontologies like the Drug Ontology (DrOn) and ChEBI, and they return more results than multiple SQL search strategies over the CDW, with few false positives. TMM is available for download from GitHub.
Electronic Health Records (EHR), Clinical Data Warehouses and related datasets play an invaluable role in the cohort-building phase of translational research, i.e. determining which patients to include. Data columns or fields that use consistent, unambiguous values (or codes) for patient characteristics are immediately actionable. Consider an exclusion criterion like “patients must have no recorded body mass index of 30 or higher”.
Medication orders, another common component of cohort definitions, are the subject of this paper. In the ideal system, patient identifiers would be linked to encoded medications with rich, unambiguous semantics like “Vytorin tablets contain simvastatin and ezetimibe”. That would make the medication records as directly actionable as discrete values like BMI. Free-text mentions of medications in clinical notes fall at the other end of the usability spectrum, as they require NLP techniques like grammatical parsing and named entity recognition.
This paper specifically addresses a task whose complexity lies in the middle of that usability spectrum: encoding the meaning of medication strings, like one would find in an EHR’s or CDW’s medication name column. Generally, these lists are strongly enriched for ingredient names, brand names, routes and or strengths, so there is great potential for processing all of those data efficiently and in bulk. If a software tool could encode the medication strings, according to a vocabulary interoperable with multiple semantic or ontological models, then additional knowledge of the drug’s therapeutic intent, side effects, etc. could be exposed.
In our CDW, the MEDICATION table is a heterogeneous mix of signals like those listed above, along with noise, like devices, medical supplies, nutritional products, lab orders, patient notes, etc. While the majority of heterogeneity and noise appears to come from legacy data in the CDW, it can also be observed in data loaded from the current production EHR, which presumably encourages the use of pick lists. Additionally, the drug orders in our CDW come from health care encounters with > 100 types, including emergency, inpatient, outpatient, office visit, phone visit, etc.
In cases like this, heterogeneity and noise can lead to the use of regular expressions
and value sets, both of which can be difficult to maintain or properly attribute. Consider
searching for patients with orders for HMG-CoA reductase inhibitor drugs from the same
structural series as mevastatin, or “people taking statins” [
The CDW also provides PHARMACY_SUBCLASS annotations for a subset of the medications. Unfortunately, an SQL search that takes a more semantic approach by requiring a PHARMACY_SUBCLASS value of ‘HMG-CoA Reductase Inhibitors’ only returns 207 drugs. Table 1 shows the proportion of drugs annotated with the top five PHARMACY_SUBCLASSes for both an unqualified ‘statin’ substring search and for the search with several negations.
In the PennTURBO initiative [
Determining which prescribable products (in a brand or ingredient sense), are mentioned by the medications modelled in the EHR/CDW Determining which drug ingredients belong to the statin class, in terms of their structure or biochemical activity
Step 1 is largely lexical, or based on the character composition of different strings. Those comparisons could be made based on rigid rules, like the SQL queries above, but TMM uses a more flexible machine learning approach. Steps 2 thorough 4 are more semantic in nature, i.e. concerned with entities that exist in the universe (of medications, in this case) and the ways in which they are related.
There are numerous tools [
Unfortunately, the lexical technologies in many of these tools require either the software developer or the user to maintain a dictionary derived from one or more authoritative knowledgebases like RxNorm. There may also be a statistical model for aligning the medication orders with the dictionary and setting cutoff scores or confidence values. These maintenance requirements may cause the software tool to lag behind the approval and availability of new drugs. And even if these tools map a peculiar medication string to an RxNorm CUI with moderate confidence (because a perfect match isn’t present in RxNorm), there may be no explanation for why the score is only moderate. How can we model the fact that a real patient took a real drug that just doesn’t happen to be modeled by RxNorm? Perhaps we can describe the way their drug is related to an RxNorm entity, using RxNorm’s own language.
TMM mappings are built with lexical and semantic approaches; they report
confidence from quantitative and semantic standpoints; and they enable users to access
the modeled data through lexical and semantic channels. The lexical matching is
performed against all the labels of all RxNorm term types, but the semantic portion
enables searching against realism-based knowledge in OBO-foundry ontologies like
ChEBI ([
TMM requires no annotated, source-specific “gold-standard” training data, although it can accept and generally benefits from normalization rules. By normalization, we mean replacing source specific language in the EHR (“po tabs”) with RxNorm language bearing the same meaning (“oral tablet”). The generation of these normalization rules is currently largely manual, and the rules themselves are stored in a CSV configuration file. Our normalization can also drop several of the tokens from “ALTEPLASE (TPA) IV BOLUS FOR ACUTE ISCHEMIC STROKE” as they are noise for our purposes: they don’t say anything about the drug product itself.
TMM explicitly anticipates that, for many of the medication strings, it won’t be possible to map to an RxNorm with perfect fidelity. In most cases, each medication string is initially mapped to multiple RxNorm terms. For each RxNorm mapping, the probability that the RxNorm term perfectly captures the detail of the medication is reported. Additionally, TMM reports the probability that an RxNorm mapping is off by one well-characterized relationship in the RxNorm graph. Imagine a patient obtained 565 mg atorvastatin tablets while travelling outside of the US. When the string “565 mg atorvastatin tablets” is processed by TMM, an exact match will not be found in RxNorm, which only models drugs approved for sales in the US. Nonetheless, TMM will still map that string to multiple RxNorm terms, including “atorvastatin Oral Tablet”, RxCUI 370621. One could represent confidence about the mappings with tuples like {(“identical”, 0.01) (“has ingredient”, 0.01) (“has dose form”, 0.95) (“more distant” 0.01)…} TMM also includes methods for electing the best of the multiple mappings.
Where possible, TMM uses existing tools and strategies instead of reinventing the
wheel. Our choices for components were largely influenced by commitments to fast,
robust performance and especially to maintainability. Medication-specific search tools
like RxNav [
In remainder of this paper, we will describe • • • •
Patterns in a CDW that could make medication mapping challenging, with examples from an otherwise high-quality CDW commonly used at the University of Pennsylvania TMM methods for building a semantic graph that includes knowledge about patients, medication orders, and medications in general Methods for searching the TMM graph from text-relevance and semantic perspectives
TMM applies the previously described normalization to medication strings and then submits them to the RxNav search engine. RxNav returns matches, with scores based on the Jaccard index between the tokens in the input and those in in the matches. Then TMM adds features like multiple string similarity measures between the inputs and matches (lexical), rxcui.count (a relevance metric, specifically the number of other medication strings that share the same RxNorm match), the semantic type of the matching RxNorm term (TTY, like “ingredient”, “brand name”, “dose form”), and the upstream source from which RxNorm obtained the match’s label or synonyms (SAB). These features are used as inputs into a random forest (RF) classifier that simply learns how to interpret the features as predictors of the semantic relations between searches and matches (“has ingredient”, “brand name of”, etc.). TMM also imports additional knowledgebases and builds a Solr index, so that users can query the data with lexical and semantic approaches, akin to the construction of the overall knowledgebase. Prototypical queries retrieve or count patients whose medication orders follow patterns like: contains an ingredient like “atorvastatin”; branded, with a name like “Lipitor”; contains an ingredient that belongs to a structural class like “statins” or bears a biochemical role like “HMG-CoA reductase inhibitor”
We have been granted access to an Oracle-based CDW that integrates records from Penn’s current and legacy EHRs. All medications known to the CDW are modelled in a MEDICATION table, which has columns such as a Primary key, a foreign key to the ORDER table, FULL_NAME, GENERIC_NAME, and a sparsely filled RXNORM. By joining the ORDER table’s foreign key to the ENCOUNTER table (which contains a patient identifier column), it is possible to count the number of patients for whom any given MEDICATION was ordered. In order to improve TMM’s performance and to keep protected health information (PHI) out of the final product, we don’t process any MEDICATION that was ordered for fewer than two distinct patients.
As of early May, 2020, the MEDICATION table had 948k records. The most commonly ordered MEDICATION is “ACETAMINOPHEN 325 MG PO TABS”, which has been ordered for 308,946 unique patients. However, most medications have been ordered for a very low number of patients: 132,748 medications ordered for only one patient, and 700,117 medications tied to zero patients. Among the 115,573 MEDICATIONs ordered for 2+ patients, 28.4% are already RxNorm-annotated. Of those, 2,127 are no longer present in the 2019AB release of RxNorm. Therefore, TMM uses an RxNorm-vs-RxNorm self-training approach, instead of using the source RxNorm terms as a gold standard for training the RF.
TMM is implemented with several scripts in the R language (tested under versions 3.5
though 3.6). The current version has been executed on Ubuntu Linux 18 LTS and Apple
Mac OS Catalina systems. TMM is unlikely to work on any system with less than 32 GB
RAM. See supplemental materials for required R libraries and additional details. [
TMM also requires Solr (tested with 7.3.1), a local installation of the
RxNav-in-abox Docker environment, and a Docker engine compatible with docker-compose, with
one small configuration change. [
TMM is designed to use an RDF triplestore database to hold its results, along with RDF serializations of the core public datasets. Specifically, TMM has been designed to use OntoText’s GraphDB. The process of querying TMM or submitting updates via some other triplestore is completely possible with minimal modification, but the process of loading the core public datasets and a RDF representation of the mapping process could be significantly different with other triplestores. We have used GraphDB versions 8.x through 10.x. GraphDB requires Java 1.8. The free version of GraphDB is suitable as long as only one user will be submitting queries or updates at a time.
Internally, TMM generates a tabular representation of source medications, search
results and classifications. Because TMM uses a triplestore as durable storage, a
conversion to RDF is required. Several libraries are available for converting tables to
RDF in R, but we have not been satisfied with their performance when it comes to
generating triples from millions of rows by 50 columns. Therefore TMM exports the
tables in a form that is compatible with the Java-based ROBOT [
TMM is available for download, with documentation, from GitHub [
Once the dependencies have been installed, users should create a table with the medications from their system in TMM’s format (Table 2). We recommend writing a custom R script that queries their EHR or CDW and saves the results as source_medications.Rdata in R’s binary format, in order to avoid problems with character encoding or with special characters, like quotation marks, carriage returns and other non-printing characters.
For the previously described string normalization, TMM also requires a
med_name_normalization.csv file with five columns, three of which are parsed
(Table 3). Users can store notes in the additional mask and Notes columns. A template
with examples is provided [
We agree that some tokens need to be excluded and that automated generation of an exclusion list is an unsolved problem. Fortunately, the extent of this problem is dramatically decreased in our case by the two patient minimum. Beyond that, med_name_normalization.csv can be used to declare both “drop any appearance of ‘ACUTE ISCHEMIC STROKE’” and “replace ‘po tabs’ with ‘oral tablet’”. Algorithm
rxnav_med_mapping_proximity_training_no_tuning.R is used to train TMM’s RF. Like most of these R scripts, it begins by importing rxnav_med_mapping_setup.R, and parses rxnav_med_mapping.yaml. A template for this file is provided but users must enter site-specific details. This script begins by retrieving a random number of medication labels from RxNav, which are lowercased, uniquified, normalized, and submitted to RxNav’s approximateTerm REST endpoint. RxNav reports identifiers for the matches, but not strings. String distances are a valuable feature for training the RF, so we obtain the match strings, the upstream source (SAB) and the string type (TTY) with a subsequent SQL query against RxNav in a box. rxnav_med_mapping_proximity_training_no_tuning.R. TMM also calculates the number of characters and space-delimited words in query and match strings, along with the total number of times each RxCUI appears in the search results.
The matches’ RxNorm identifiers stand in place of medications that exist in our universe and can share relations with one another, like “has brand name” and “is ingredient of”. Because we want the TMM RF to predict relations between inputs and search results, another SQL query is submitted to RxNav in order to retrieve all of the relations between all of the entities known to RxNorm. When the input and a search result are semantically identical, TMM asserts the non-RxNorm relation “identical”. When no direct relationship is observed between the search term and a match, the nonRxNorm relation, “more distant” is asserted. The RF is trained after splitting the feature rows and known relationships into training and validation subsets. Then it is saved, and assessed with a typical multi-label confusion table. We also provide an example script for interactive tuning of the RF.
Up to this point, the “unknowns” and the “gold standards" relations have both come from RxNav, so the “unknowns” haven’t required any normalization. Next, rxnav_med_mapping_proximity_classifier.R is used to normalize the FULL_NAMEs from source_medications.Rdata according to the patterns in med_name_normalization.csv. After this, they are to the approximate match REST endpoint and additional features are generated as above. The RF is applied to those features in order to predict the relations between the unknowns and the search results. After applying the RF, each search result will have one confidence score for each of the relations defined by RxNorm, along with “identical” and “more distant”. Even though one relation will have the highest probability for each search result, the number of classified search results for each input could be zero to 50. A low pass, destructive filter is applied at this point: if there are any “identical” results, only the one(s) with the highest probability are retained. If there are no “identical” classifications, then the results classified with a single RxNorm relation are evaluated, and regardless of the relation type, the one with the highest score is retained. Finally, in the cases where all of the search results for a given input are classified as “more distant”, the one(s) with the highest score for anything other than “more distant” are retained. After this filtering, the R script exports its internal tabular data structures, which are converted into RDF with ROBOT.
NCBO BioPortal term mappings in the core public ontologies should be retrieved and converted to RDF with serialize_bioportal_mappings.R. These mappings support our SPARQL queries in cases where there are gaps in the axioms provided by RxNorm, DrOn and CHEBI.
rxnav_med_mapping_load_materialize_etc.R loads each of the previously mentioned RDF files into an isolated named graph in GraphDB, which scopes queries and improves their performance. This script implements other performance improvements by executing several SPARQL updates that transitively materialize subclass relationships and convert complex OWL restrictions into shortcut relations like “has ingredient” and “bears role”, also into isolated named graphs.
The scripts described above generate 18 features that could be used to train a random
forest [
With the exception of “more distant”, all of the classifications, including “identical” have > 0.95 specificity (Table 4). “Identical” has reasonable sensitivity (0.822), but the sensitivities of the other classes defined by an RxNorm relation only have an average sensitivity of 0.35. Conversely, “more distant” has excellent sensitivity (0.973) but lower specificity (0.755). In fact, almost all of the misclassification can be attributed to overinclusion in “more distant”. Finally, this script addresses the fact that a medication string could still have multiple mappings, even after the low-pass filter, by electing the RxNorm term whose sum of “identical” scores is highest. Note that the prevalence of the different RF classifications are very uneven, they are consistent with the training data. ‘contained in’ relationships, which describe the make-up of a drug package, are just rare in the universe as modelled by RxNorm.
A Solr query was run in < 1 second to determine that the “statins” structural class is
best modelled as <http://purl.obolibrary.org/obo/CHEBI_87631>, and a SPARQL query
was run in < 2 seconds to find 1,002 medications in our CDW with a path back to
<http://purl.obolibrary.org/obo/CHEBI_87631>, compared to the 610 statins found by
an SQL statement based on substring searching with negation [
The CDW contains 115,752 “common” medications strings, meaning that they were ordered for at least two patients. TMM mapped 108,995 of those to some RxNorm term. Table 5 shows details about the accuracy of the TMM semantic search over the random forest results, plus three SQL queries described over the course of this paper. Figure 1 is provided as background regarding the number of patients with orders for any given medication, as the impact of those counts is addressed along with the accuracy. TN
Sn
Sp
F1 114719
The intersection and disjoints between TMM’s semantic statin search and the most semantic (although least productive) SQL search for ‘'HMG CoA Reductase Inhibitors' were manually evaluated. 136 of the common medications were identified both as HMG CoA Reductase Inhibitors in the CDW and as statins by TMM. 'ATORVASTATIN CALCIUM 40 MG PO TABS' was ordered for the greatest number of patients: 63,334.
Ten common medications were annotated as HMG CoA Reductase Inhibitors in the CDW but were not classified as statins by TMM. For 9 of the 10 medications missed by TMM, it was RxNav that did not find any matches. In all of these cases, manual review showed that the medication was not modelled in the current RxNorm. The most common medication in this category was 'BAYCOL 0.4 MG OR TABS', which was ordered for 141 patients. Relative to the CDW, TMM's only outright false negative for statins was 'LOVASTATIN 10 MG OR TB24', with 95 orders. The fact that “TB24” means “24 hour tablet” was absent from the normalization file, so that could have degraded the mapping.
It appears that 13 of the TMM only identifications were false positives. '.Amlodipine Tablet' was the worst offender with 5727 orders. 'niacin ER -' was ordered for '1013' unique patients. We believe that these 'statin' misclassifications might be attributed to the random forest trainer's exposure to medications in which those compounds appear along with statins in combination drugs. Examples of other misclassifications included '.Sodium Hypochlorite Soln0.13% (1/4 Stg)' with 7 orders, 'HYALURONATE SODIUM (EMOLLIENT) 0.1 % EX LOTN' with 3 orders, and 'Please have a Basic Metabolic Panel and Magnesium and Phosphorus level drawn on Thursday (1/12), Monday (1/16), Thursday (1/19), and Monday (1/23).' with 3 orders.
866 medications were identified as statins by TMM but were not annotated as HMG CoA Reductase Inhibitors in the CDW. The most common medication in this category was 'atorvastatin -' with 40,937 orders. Similar patterns were observed for 'pravastatin -', 'rosuvastatin -' and 'rosuvastatin -'. Those are naturally found by the best SQL search (‘statin” substring, with negation). Of the 866 TMM-only statins, 811 had been imported into the CDW from a legacy system and 55 were loaded from the current production EHR. Overall, 59737 orders were from the legacy system and 56015 orders were from the production system.
We envision several ongoing improvements to TMM. The number of “more distant” mappings could probably be decreased with paths between medication strings and RxNorm terms over two RxNorm relations, like “ezetimibe 10 MG / Simvastatin 10” has_tradename “Vytorin Pill” has_ingredient “Vytorin”. However, there are 26 RxNorm relations, so the cartesian product might result in an impractical number of paths and decreased performance. RxNorm has monthly full releases in the UMLS’s Rich Release Format (RRF), but we have been getting an RDF version from the BioPortal, which only has biannual releases. We are currently developing our own RxNorm RRF ® RDF converter.
We have described methods for filtering and electing mappings, so that only the highest scoring matches are retained. In rare cases, multiple mappings with the same score still persist. The election process could possibly be improved by looking for the most central node (from a graph perspective) or a lowest common subsumer (from a hierarchical perspective.) We could also check that the RxNorm term is even compatible as the object of the predicted relationship. For example, ingredients can’t “have ingredients”.
Some of our clinical collaborators have observed false positive, false negative or generally under-specific role and class mappings in the public ontologies we import. Fortunately, the maintainers of the public knowledgebases we have mentioned are receptive to feedback leading to updates. We would still like to better characterize therapeutic indications (mechanism of action, physiological effect, etc.) present in the ATC and NDF-RT subsets of UMLS, although their semantics don’t have same rigor as OBO foundry ontologies. We also plan to add a systemic method for rapidly capturing our collaborators’ alternative classifications, naturally using named graphs.
When applied to our CDW, TMM finds more “statin” medications than SQL queries over the medication strings, the sparse pharmaceutical classifications or the sparse legacy RxNorm classifications. When we have done deep dives on other medication classes like analgesics (opioid or nan-narcotic), we have found the same result [data not shown]. The relationships between these classes, ingredients and products are backed with sound semantics.