We propose the use of vector-based word embedding models to learn a cross-conceptual representation of medical vocabulary. The learned model is dense and encodes useful knowledge from the training concepts. Applying the embedding to the concepts of diagnoses and medications, we then show that they can then be used to measure similarities among patient prescriptions, leading to the discovery of in- formative and intuitive relationships between patients.
In simple word representation techniques such as the Ngram model [Brants et al., 2007], words are regarded as single atomic units, and no notion of similarity between words exists. Conversely, distributed word representations in vector space provide an explicit grouping of similar words to achieve high performance in Natural Language Processing tasks [Rumelhart et al., 1988]. Such embeddings rely on vector operations to represent learned word proximities or similarities [Mikolov et al., 2013] and have have been used to efficiently learn high-quality word vectors from very large datasets (containing billions of words) using a vocabulary containing millions of words [Collobert and Weston, 2008; Bengio and Usunier, 2011; Socher et al., 2011; Glorot et al., 2011; Turney and Pantel, 2010; Turney, 2013].
Recently, [Mikolov et al., 2013] has introduced a neural network design using distributed word representations to capture interesting features such as linguistic regularities and patterns. The architecture, named the Skip-Gram model, is trained to find word representations of a given (input) word that are useful in predicting its surrounding words in a sentence or a document. The vector representation used in the Skip-Gram model highly increases the network’s training efficiency, with the ability to train 100 billion words in single optimized machine [Mikolov et al., 2013].
Our idea lies in using a Skip-Gram model to learn a compact representation of patient features. Using an initial model with medications and diagnoses as features, we propose a scheme to embed top-level ICD 9 codes of patient prescriptions and diagnoses within the same continuous representational space. We then build a skip-gram representation using the chosen system to create a compact and continuous representation of patients enabling: 1) efficient feature processing and 2) some degree of generalization in finding similarity between patients given their features. Using our model, we would be able to reach the natural conclusion of a patient diagnosed with Diabetes being similar to a patient receiving insulin treatment. This is a non-trivial exercise for a machine learning algorithm, as we understand that the two cases are to some degree the same abstract concept expressed across two different domains (diagnoses vs. treatment).
The paper is structured as follows. After a brief illustration of the required background in Section 2, we discuss our architecture in Section 3. In Section 4, we show the results of training the resulting neural network model on a large database of intensive care unit medical records. We conclude with ongoing work and future directions in Section 5. 2 2.1
A well-established approach for representing concepts to facilitate learning is the use of a fixed dimension, real valued vector representing words. Each entry of this vector corresponds to some feature in a hypothetical latent space, rendering the size of the vector to be the dimensions of the feature space used to represent a single word.
For example, creating a 5-dimensional representation of prescriptions such as Aspirin, Ibuprofen, and Insulin, we could decide on features such as “Heart problems,” “Pain killer,” “Kidney Problems,” “Critical Importance medication” and “Preventative treatment.” In this example, Aspirin would rank moderately for “Heart,” quite highly “Pain killer,” relatively lowly for “Kidney Problems,” perhaps low to moderately for ”Critical Importance” and moderately to high on “Preventative.” Normalizing the values of an arbitrary patient (by assuming a vector length of unity) gives the vector shown in Table 1.
In practice, we do not suggest the nature of each feature, but merely supply the number of them - a neural network or another approach then learns these features so as to serve its needs best. However, the basic premise is the same - each feature has some meaning in the hypothetical latent space learned by the network, and so similar values in the same position indicate two samples both share some aspect of this feature. Examples which share a large number of features are therefore closer than those which share only a few, as a consequence of this encoding, which is the mechanism by which similarity is explicitly encoded as Table 2 shows. The Skip-Gram model is based on the goal of finding word representations that would enable the prediction of surrounding words of a given word in a sentence. The idea is for any ’candidate’ word found in the training vocabulary; we can associate the most likely ’context’ word such that the two words show the maximum association.
Formally, given a sequence of words w1, w2, ..., wn, ..., wN , the Skip-Gram model will train a multi-class logistic regression so that for each candidate word wn, we can find a ’context word’ wn + j falling within the window of c words before or after wn such that the probability of P (wn+j |wn) is maximum [Mikolov et al., 2013]. In other words, the Skip-Gram model aims to maximum the average log probability: 1 XN N
X n=1 c j c,j6=0
logP (wn+j |wn) c is the size of the training context and is used to adjust the model. Larger c values associate a wider context with a given candidate word, implying more training examples, slower training but better classification. 3
The work performed here is based on the idea of generalising vector-based embeddings to any number of medical concepts, regardless of whether or not they come from the same underlying distribution. The main implication of this is that the features potentially become more general or invisible to us. However, with related domains such as diseases and drugs, we could imagine a normalized encoding as given in figure 1 for features spanning the two concepts of disease and medication. 3.1
The details of our implementation are largely based on the skip-gram model [Mikolov et al., 2013; Rumelhart et al., 1988] and is shown in Figure 1. The implemented logistic regression classifier receives as input an ID corresponding to an item in our vocabulary (in this case a list of all the ICD 9 codes for diagnoses and Medications). This ID corresponds to the Drug Embedding, which is a row within our Embedding matrix. Using the embedding lookup(...) functionality in tensorflow, we retrieve the 100-dimensional Embedding for the input, multiply it by a weight vector and pass through a softmax function. The input-output pairs are created to be all permutations of pairs that appear together in the same set. For example, if a patient was prescribed medications A, B, and diagnosis D, we create input-output pairs as: (A, B), (A, D), (B, A), (B, D), (D, A). We aggregate all these input-output pairs across all patients in the training set and use them to perform mini-batch back-propagation on the embedding matrix and logistic regression parameters simultaneously. As proposed by Mikolov et al. [Mikolov et al., 2013], we use Noise Contrastive Estimation to approximate the loss at each step of training, to improve the efficiency of computation, and built our model in tensorflow. 3.2
Using the unsupervised joint embeddings, we show that meaningful patient similarities can be discovered within the data. To do this, we train the prescription and diagnosis joint embeddings in the manner described in the previous section, on a subsection of the data (100,000) prescriptions, and then draw patients randomly from the remaining portion of the data. We then aggregate all the prescriptions given on a daily basis to the patient during their stay and replace each one with the relevant embedding trained previously.
To generate a treatment vector, we average all the individual drug embeddings for each day. By then taking a single days treatment vector, and computing the cosine similarity between that and other daily treatment vectors, we can find the similarity between patient treatments.
This is a well-established trick in NLP and is often a primary benchmark to compare other methods against, and while it may not seem like the most sophisticated solution, it can be surprisingly effective. 4 4.1
The model was trained using the MIMIC dataset [Johnson et al., 2016]. This is a large Intensive Care Unit (ICU) dataset containing the records of over 40,000 patients in the ICU of the Beth Israel Deaconess Medical Center, Boston, Massachusetts, U.S.A. between 2005 and 2012. Prescriptions are registered alongside a unique and anonymized patient identifier, with a date range indicating the period this was to be administered over.
To train a neural network on patient prescriptions, one must first extract the data and reshape it, which is a non-trivial task for the way the data is presented in MIMIC iii. As shown in Figure 2, prescriptions are primarily indicated by a combination of hospital admission id, start date, end date and drug. 4.2
The first step was to aggregate all the drugs by day and hospital admission id, to compile a list of concepts to be used per day, as shown in Figure 3. Each day defines a context window for that patient, so if a patient received drugs A, B and C on a given day, the input output pairs for the network are (A, B), (A, C), (B, A), (B, C), (C, A) and (C, B).
Next, we assign each concept an arbitrary ID, with 0 reserved for an ’unknown’ entry. This allows unseen concepts to be included after training time. Each ID maps to a row in a randomly initialized embedding matrix, which has dimensions (number of drugs x embedding size). This embedding matrix is then used as inputs to logistic regression classifier, which performs a one-hot prediction for the output concept, with size (number of drugs,). This is displayed mathematically in 2.
E = embedding lookup(X) yˆ = sof tmax(E · W + b) (1) (2)
This system is trained via back propagation, and it simultaneously learns both the W and b parameters and the values of the embedding matrix. Once training is complete, the embedding matrix acts as a lookup dictionary - to get the representation for a particular drug, simply find the ID it maps to and extracts this row from the embedding matrix. I used the standard Adam as the optimization method and negative log likelihood for the loss function.
In tensorflow, we initialized the weights randomly, with a truncated normal distribution for weights and a random uniform for embeddings and biases. This is based upon conventional methodologies found to be most useful in a wide range of settings, as described in [LeCun et al., 2012].
Following from [Mikolov et al., 2013], I use Noise contrastive estimation to improve the efficiency of the model. As the model has many outputs (one for each entry in the ’vocabulary’), computing the softmax at each stage is computationally expensive. As most of the entries are in fact not relevant (we have many classes, but most should be 0, and we want only a single entry that is substantially non-zero), we can improve the computation efficiency by sampling the loss function rather than computing it exhaustively. There are two ways to achieve this in practice, one is with a sampled softmax, which essentially computes a Monte Carlo estimate, and Noise contrastive estimation which picks examples of the positive and negative classes so as to get an estimate that way. 4.3
As this is an unsupervised approach, quantitative evaluation of the results is difficult. To assess if the neighbourhoods are correct, most previous work either appeals to experts to evaluate the quality or avoids this altogether and leaves the reader to judge for themselves [Mikolov et al., 2013].
To provide a qualitative evaluation of the results, we took the top occurring drugs and found the nearest neighbours to them using cosine similarity as a measure.
These nearest neighbour relationships show some useful similarity between drugs. For example, we see salts and electrolytes naturally grouping together (e.g. Potassium Chloride and Magnesium Sulfate). Aspirin is close to two statins drugs which try to treat blood pressure and alleviate the risks of heart attack or similar problems. Metoclopramide is used
Potassium Chloride Morphine Sulfate Docusate Sodium Calcium Gluconate
Aspirin Metoclopramide Amiodarone HCl Heparin Sodium
Magnesium Sulfate
Acetaminophen Sodium Chloride 0.9% Flush Potassium Chloride
Simvastatin
Ranitidine D5W (EXCEL BAG)
Warfarin
Calcium Gluconate Oxycodone-Acetaminophen
Acetaminophen Magnesium Sulfate
Atorvastatin
Nitroglycerin Phenylephrine HCl
Ibuprofen to treat acid reflux, a stomach complaint, and Ranitidine is used to reduce the amount of stomach acid produced.
We also see relationships between items that often appear together even if they are not direct replacements. For example, Amiodarone HCL is an antiarrhythmic drug, used to treat issues with irregular heartbeats, and its nearest neighbour is D5W. D5W is a code for Dextrose 5% and water, which is essentially just a carrier for IV lines and similar methods of delivery. These two are near as it is common within the data to administer Amiodarone HCL as a solution with D5W.
As with the prescription only embeddings, proving these encode useful information in a quantitative way is somewhat complicated. We follow the same approach as the previous section and provide some of the nearest neighbours for common entries in the data, and also, in the next section, show that these embeddings are useful for the task of finding patients with similar treatments, as a way to demonstrate that they encode relevant information.
As can be seen in table 4, the approach of using joint embeddings encodes the same relevant information seen in the results for single embeddings, while also providing links between diagnoses codes and drugs. For more broad ranging drugs, such as painkillers, we see a clustering that is not particularly associated with a single ICD9 code, for example, Bisacodyl is close to Docusate Sodium and Morphine Sulphate. This also shows another interesting artifact of this method - docusate sodium is not a painkiller, but is ’close’ to bisacodyl because they often appear together. Acetaminophen, Meperidine, and Morphine Sulfate are another cluster of pain relief medications which do not appear ’close’ to a particular ICD9 diagnosis code.
We see interesting clustering of ICD9 codes - 427, 428 and 414 all representing heart problems for example. We also see cross group clusters, which put Diabetes and Insulin close together, as well as Aspirin and heart disease.
Finding patients who shared a similar daily treatment vectors worked well to find patients of similar types. Due to the nature of the ICU, many patients received a large number of drugs, and using embeddings rather than a one hot style approach allows for meaningful entries to be more discriminative. We selected patients at random, and then picked a random day for that patient, and computed the cosine similarity between that daily treatment vector and all other daily treatment vectors for all patients. As expected, other days from that patients stay in the ICU rank very highly in many cases. However, even if we look only at other patients, we see meaningful groupings occurring. Some examples are included in Table 5. Similarities are Cosine similarities of normalised vectors, and so they vary between 100% and -100%. A similarity of 100% means the same, while -100% indicates 276 (Disorders of fluid electrolyte) 530 (Diseases of esophagus)
Bisacodyl Calcium Gluconate
Acetaminophen
Insulin 427 (Cardiac dysrhythmias) 401 (Essential hypertension)
Metoprolol 250 (Diabetes mellitus)
Docusate Sodium
SW
Meperidine 250 (Diabetes mellitus) 428 (Congestive heart failure) 746 (Other congenital anomalies of heart)
Clopidogrel Bisulfate
Acetaminophen
Guaifenesin Chlorhexidine Gluconate
Cefazolin
Insulin complete opposites. As such, we can show example pairs which are similar, and those which are also not similar.
As shown in Figure 4, if we conduct PCA on the 100dimensional embeddings to reduce the dimensionality down to two principle components, we can visualise the ’closeness’ of a small number of entries. The figure shows that the selected entries fall into 3 or 4 clusters. Diabetes is close to Insulin, which two neonatal drugs (denoted by NEO*) are close to the diagnosis ICD 9 code for electrolyte imbalance - a condition that strongly associates with newborns in the dataset. The other cluster corresponds to pain medication (and docusate sodium which is used to alleviate constipation, a common side effect of many pain medications). This cluster seems to split into two subclusters, potentially corresponding to differing use of the different drugs, but likely an artifact of the dimensionality reduction. 5
The idea and initial results presented here are part of our ongoing work of finding compact representations of patients using their electronic hospital records, to make use of the massive deluge of longitudinal data available to understand the densest set of features representing a given patient. Our long-term goal is to complement endeavors in personalized medicine by devising patient similarity measures that can be used to supplement scientific inquiry and which can translate into actionable knowledge in the bedside. For instance, we would like to be able to answer questions such as Given that patient X is similar to patient Y, will X respond to treatment Z similarly to patient Y ? Or why has patient A responded differently than patient B to the same treatment provided?
The authors would like to acknowledge the supports of the NIHR Biomedical Research Centre for Mental Health, the Biomedical Research Unit for Dementia at the South London, the Maudsley NHS Foundation Trust and Kings College London, and a joint infrastructure grant from Guys and St Thomas Charity and the Maudsley Charity, London, United Kingdom.
This research was also supported by researchers at the National Institute for Health Research University College London Hospitals Biomedical Research Centre, and by awards establishing the Farr Institute of Health Informatics Research at UCLPartners, from the Medical Research Council, Arthritis Research UK, British Heart Foundation, Cancer Research UK, Chief Scientist Office, Economic and Social Research Council, Engineering and Physical Sciences Research Council, National Institute for Health Research, National Institute for Social Care and Health Research, and Wellcome Trust.
Newborn Stab Wounds Pneumonia
Diaherria Urinary Tract Infection
Encephalopathy Incarcerated Hernia Newborn
Newborn Motor Vehicle Accident
Hypoxia
Fever
Renal Failure Congestive Heart Failure
Sepsis Arterial Injury
Treatment Similarity 99% 89% 85% 85% 78% -41% -40% -35% In International conference on Machine learning, page 160167, 2008.