Among deep learning approaches, the Long Short-Term Memory (LSTM) networks have been observed as one
of the most e ective solutions to deal with MTS [HS97a]. LSTMs present several advantages over conventional
feed-forward and recurrent neural networks. These advantages include the capacity of LSTM to grasp the
longterm dependence of time sequences, which provides them with great versatility. LSTMs have been traditionally
used in tasks related to language modeling, image captioning and translation, among others [YPL+17, SSN12].
The use of LSTM in areas such as medicine has signi cantly increased during the last years. For example, Lipton
et al. applied a LSTM network to classify diagnoses based on the temporal data recorded in the Electronic
Health Record (EHR) of a pediatric Intensive Care Unit (ICU) [LKEW15]. Nguyen et al. used a bidirectional
LSTM model with an attention mechanism to predict ICU mortality outcomes, showing co
In this paper, we deal with these challenges by exploring the potential of the LSTM networks to predict multi-drug resistance (MDR) in the ICU. MDR can be de ned as the capacity of bacteria to withstand the e ects of a variety of harmful chemical agents designed to damage it [MDHL14]. That means that diseases easily treatable today will require much more complex and expensive treatments than the current ones. Eventually, they could even suddenly become lethal. Even more everyday situations such as minor injuries or cuts could pose a very high risk [MDHL14]. To know the in vitro activity of an antibiotic against a given bacterium (previously isolated in the culture), the results of the antibiogram are analyzed. These results usually take between 24/48 h. Accurate and faster identi cation of the MDR may enable to implement preventive steps, patient isolation, and therefore a reduction in the rising rates of MDR.
Speci cally, we analyze MTS measurements of mechanical ventilation as well as the use of di erent families of antibiotics by the patients at the ICU of the University Hospital of Fuenlabrada (UHF) in Madrid (Spain). Taking into account these series, we propose to predict the MDR imminence by applying a LSTM network.
The rest of the manuscript is organized as follows. Section 2 presents the notation and the statistical approaches used in this paper. In Section 3 we describe the data set, while experiments and results are shown in Section 4. Finally, main conclusions and discussion are drawn in Section 5. 2
In this paper, the learning algorithms are performed on a set of D time series per patient, with one temporal series per variable. Each time-series is composed by T time slots. Thus, data associated to the i-th patient can be arranged in the matrix Xi = [xi1; xi2; : : : ; xiT ] 2 RD T . Column vector xit contains the D variables associated to the t-th timestlot, i.e., xit = (xi(t;1); xi(t;2); ; xi(t;D)). Thus, xi(t;d) represents the value of the d-th variable in the t-th times slots for the i-th patient. Since we are interested in a binary classi cation problem, we have considered the label `1' to identify patients with MDR, and the label `0' to identify patients with non-MDR. Thus, we represent the label for the i-th patient by yi, and the output provided by the network with y^i. 2.1
In this paper we will explore two conventional methods: logistic regression (LR) and voting k-nearest neighbours (k-nn). On the one hand, Logistic Regression (LR) is a parametric approach based on a logistic function to estimate a binary variable as a linear combination of d independent features [DGK02]. The coe cients of the linear model are found by optimizing a regularized cost function, which depends on a penalty coe cient C, to prevent over tting (Ridge regularization [Tik63]).
On the other hand, k-nearest neighbour (k-nn) is a non-parametric and non-linear classi er well-known in the literature. This classi er depends on a distance measure to determine the similarity between samples. In this paper we have considered the Jaccard distance, which is widely used in the literature for binary variables [Ouy16]. The voting k-nn assigns a class to an unclassi ed sample taking into account the majority class among the k closest neighbours of the sample in question. Note that the appropriate choice of k conditions the behaviour of the classi er. It is highly dependent on the dimensionality and the training set size, and should be chosen following a cross-validation strategy. The LSTM architecture is a special kind of arti cial neural network (NN) composed by a set of recurrently connected subnets, knowns as memory blocks [HS97b]. The main di erences between the architecture of standard NN and LSTM are that, on the one hand, conventional NNs can only map from input to output vectors and, on the other hand, conventional NNs do not allow cyclical connections between neurons. As a consequence, standard NNs are not capable of mapping sequential inputs.
When NNs consider cycles between neurons, the architecture is named as Recurrent Neural Network (RNN). Note that cycles between neurons allow preserving as an `arti cial memory' the information of previous inputs in the internal state of the RNN. Therefore, RNNs can be considered as NNs specialized in mapping sequential inputs [Gra12]. Unfortunately, it is not convenient to work with standard RNN architectures when considering very long MTS. The problem occurs due to the in uence of past inputs on the hidden layers because of the cycles between the neurons and the arti cial memory commented previously. If the arti cial memory makes the gradient either decays or blows up exponentially, the RNN does not work properly. This e ect is referred to as the vanishing gradient problem [Gra12]. The LSTM invention was motivated for avoiding the gradient's problems of the RNNs.
The key element of LSTM networks is the cell state (see Fig. 1), which considers information of previous inputs [Gra12]. A simple LSTM cell consists of three kind of gates: forget, input and output.
The forget gate layer (f t) decides what information of the past states is going to be deleted. It uses ht 1 (the output of the previous state) and xt, then the result passes through the sigmoid activation function. This process creates a mask that multiplies the previous state St 1.
The input gate layer (St) determines what new information (from xt) is going to be stored in the cell state. This layer is composed of two mechanism, the rst one creates a vector of new candidate values, S~t and the second one, it decides by how much we decided to update each state value. Next, it and S~t are multiplied and the result is added to the previous St 1 f t 1.
The output gate layer (ot) decides the exact output, which will be based on our cell state but will be a ltered version. It is composed of a rst sigmoid sublayer which decides what parts of the cell state will be output. This creates a vector known as ot. Then the cell state St pass through a tanh function (to push the values in the interval [-1, 1]) and multiplies ot giving as a result ht. 2.2
As it usually happens in the medical context, the classi cation task is highly imbalanced, with a lower number of patients with MDR regarding those with non-MDR. Dealing with imbalanced classes can lead to a poor model performance since the model can be biased to the majority class [WLW+16]. Since the LSTM network presented above does not have any mechanism for dealing with imbalanced data sets, we have considered two di erent strategies to train the model: (1) Balancing classes in training, following an undersampling approach to reduce the number of patients (samples) associated to the majority class until both classes have a similar number of patients; (2) Considering cost-sensitive learning, i.e., penalizing errors on the minority class by a cost proportional to how under-represented this class is. In the following, we assume that the network can provide estimates y^ of the posterior probability for class `1'. Thus, assuming that the size of the training set is N , the cost functions considered in this paper are presented below:
The Binary Cross-Entropy (BCE) function is one of the most used for in binary classi cation problems. For this reason, we have considered it as a baseline. The BCE function increases as y^ diverges from the actual label y as follows: 1 N
X (yi log (y^i) + (1 N i=1
The Focal Loss (FL) function reshapes the BBCE function with a tunable focusing parameter 0 (BBCE is obtained with = 0). As indicated in [LGG+17], the idea is to down-weight samples easy to classify and thus to focus training on those samples of the less frequent class which are hard to classify. The FL function is de ned as y^i) log (y^i) + (1 ) (1 yi) y^i log (1 y^i)) (3)
Regarding the gures of merit considered to evaluate the model performance, we have considered the following ones: Classi cation Accuracy, Speci city, Sensitivity and F1-score. The Accuracy is the ratio of correctly classi ed samples over all samples used for evaluation. It can provide misleading conclusions because results can overestimate the model performance, e.g. high accuracy rate can be achieved while most samples for the minority class are incorrectly classi ed. To avoid the potential bias of the Accuracy towards the majority class, Speci city and Sensitivity get the scores for each class. In particular, Sensitivity (also called Recall) has been computed as the proportion of patients with MDR who are correctly identi ed by the model as having the condition; Speci city refers to the proportion of non-MDR patients who are correctly identi ed by the model as not having the condition. With regard to the F1-score, it is the weighted average of Sensitivity and Precision (number of correctly predicted MDR-patients over the total number of patients labelled by the model as MDR-patients). The use of Accuracy is more appropriate when false positives and false negatives have similar \cost", otherwise, the use of F1-score is more convenient. At this work, we have stopped the LSTM network training according to an early-stopping approach based on the F1-score. 3
Data analyzed in this work were collected from the EHR of UHF for a period of 13 years, running from 2004 to 2016. A total of 2,540 patients were admitted to the ICU during this period, and 449 of them had MDR bacteria. A patient is identi ed as MDR if a culture is agged as positive, indicating that MDR bacteria was found. Since several cultures of the same patient can be agged with MDR bacteria, we have limited our research to the prediction of the rst of these cultures. As a consequence, we have not considered the patient's history after the time step when this culture is performed.
To have an idea of the length of the series, we show in Figure 2 the distribution of these time series both for MDR and non-MDR patients. Note that the time series for MDR patients are considered just until the day any culture is agged as positive. Figure 2 (a) shows the histogram of the elapsed time from the ICU admission to the ICU discharge for patients with no culture agged as positive along with the whole ICU stay. For patients with any culture agged as positive, the distribution of the elapsed time from the ICU admission to the time the rst MR culture agged as positive is shown in Figure 2 (b). We have tacked the patient characterization by a MTS registered every 12 hours during a 7-day window. For MDR patients, we consider the rst time slot (t=1 in X, the rst 12 hours) as the day the culture that will be agged as positive was performed and then continue completing the series in reverse time order. For non-MR patients, we also consider a reverse time order in matrix X, but the rst time slot corresponds to the day the patient was admitted in the ICU. Figure 3 shows a sketch illustrating it.
In this work, the MTS are represented by the family of antibiotics taken by the patient, as well as the time the patient has been assisted with mechanical ventilation. Regarding the rst group of variables, a total of 15 antibiotic families were extracted: Aminoglycosides, antifungals, carbapenems, third generation cephalosporins, fourth generation cephalosporins, glycylcyclines, lincosamides, nitroimidazoles, oxazolidinones, broad-spectrum penicilins, penicilins, polymyxins, quinolones, sulfonamides and non grouped antibiotics. Associated to each family of antibiotics, a binary variable was created to indicate whether the patient has taken (or not) that family (a) (a) (b) (b) of antibiotics during a period of 12 hours. Regarding mechanical ventilation, a binary variable indicates whether the patient has been connected (or not) to a breathing machine for a temporary period of 12 hours.
Apart from the clinical registered variables, it has been reported that missing values are usually informative missingness [CPC+18], i.e., missing values can provide rich information about target labels in supervised learning tasks (e.g, time series classi cation). To e ectively exploit the information of missingness patterns, we will use a mask (an additional binary variable) indicating for each time slot whether the patient was in the ICU during the period of 12 hours under study or not. As it is shown in Figure 3, values in the mask are zero whenever the patient is not in the ICU. 4
The goal of the experimental work in this paper is to design a data-driven model enabling the prediction of the imminent onset of MRD for the ICU patients. Since we have information about the antibiotics and the use of mechanical ventilation along with their stay in ICU, we propose to design a LSTM network by considering MTS. Taking into account that the number of MDR and non-MDR patients are imbalanced, we consider di erent strategies to train the model and evaluate its performance. Apart from that, we also analyze the knowledge provided by mechanical ventilation when considered in conjunction with the antibiotics. 4.1
The methodology performed to train and evaluate the classi ers is the following. First, two-thirds of samples (patients) were randomly assigned to the training set, whereas the rest were assigned to the test set. A 5-fold validation strategy was carried out just on the training set to nd the most appropriate value for di erent hyperparameters. In particular, for LR we chose the penalty coe cient C 2 f0.001, 0.005, 0.01, 0.05, 0.1, 0.5%g, whereas for voting k-nn we found the number of neighbours k. Regarding hyperparameters associated to the LSTM network architecture, we considered the number of neurons in the hidden layer (with f5, 7, 10g as explored values) and the dropout rate (f0.2, 0.35, 0.5%g), used as a regularization technique to reduce over tting on the training data. Regarding hyperparameters associated to the LSTM network trained according to the BBCE function, the considered = f0.5, 0.66, 0.75, 0.83 0.875, 0.91g. For the LSTM networks optimizing the FL function, di erent values of f0.2, 0.4, 0.6, 0.8g and f0.25, 0.50, 1, 2g were considered. To provide statistics about the model performance, we repeat the training/test partitioning process 20 times. Speci cally, the mean and standard deviation of the following gures of merit in the test set was provided: accuracy, speci city, sensitivity and F1-score. We present in Table 1 the mean and the standard deviation of the performance provided by the LSTM networks in terms of Accuracy, Speci city, Sensitivity and F1-score. Several conclusions can be drawn. On the one hand, when considering only the use of antibiotics in the ICU in a 7-day window, the use of the FL cost function as a strategy to handle imbalanced data sets provides the highest averaged performance in terms of accuracy (66.83), speci city (68.67) and F1-score (37.99). The LSTM network used to obtain these results was trained using 5 neurons (with tanh activation function) in the hidden layer, dropout of 0.2, =0.8, and =1. It is important to remark that the highest values of sensitivity are obtained when applying the BBCE function. On the other hand, when analyzing both the use of antibiotics and mechanical ventilation jointly, the predictive results improved with respect to the use of MTS just related antibiotics. In this case, the highest values of Accuracy and Speci city are obtained also when the FL function is evaluated (84.34 and 94.05, respectively). The best sensitivity value (62.33) is provided when undersampling the training data set.
Respect to the LSTM network on the hand one, when considering only the use of antibiotics in the ICU in a 7-day window, the use of the FL cost function as a strategy to handle imbalanced data sets provides the highest averaged performance in terms of accuracy (66.83), speci city (68.67) and F1-score (37.99). The LSTM network used to obtain these results was trained using 5 neurons (with tanh activation function) in the hidden layer, dropout of 0.2, =0.8, and =1. It is important to remark that the highest values of sensitivity are obtained when applying the BBCE function. On the other hand, when analyzing both the use of antibiotics and mechanical ventilation jointly, the predictive results improved with respect to the use of MTS just related antibiotics. In this case, the highest values of Accuracy and Speci city are obtained also when the FL function is evaluated (84.34 and 94.05, respectively). The best Sensitivity value (62.33) is provided when undersampling the training data set. Training the LSTM with the BBCE function, 10 neurons in the hidden layer, dropout of 0.2, and =0.66, provides the highest F1-score (50.31). Since F1-score is the harmonic mean of precision and recall, it takes both false positives and false negatives into account. Therefore, the higher the F1-score, the better the classi cation results.
To show the potential of LSTM in this scenario, conventional approaches as LR and voting k-nn were also considered with the undersampling approach. We checked that performance provided by LR and k-nn worsen to those obtained with LSTM in terms of balancing between speci city and sensitivity. Speci cally, when considering all features we obtained Sensitivity values of 31.51 5.93 and 30.16 8.57 for LR and k-nn, respectively; whereas the Speci city values were 96.46 1.51 and 93.52 3.72. Performance of conventional approaches even worsen when considering just antibiotics (Sensitivity values, 22.35 7.32 and 26.48 9.04 for LR and k-nn, respectively; Speci city values, 96.36 1.60 and 91.47 4.98 for LR and k-nn, respectively.
Furthermore, since the sigmoid activation function is used at the output node of the LSTM network, the output can be considered as an estimation of the posterior probability for the positive class. Fig. 4 shows the output distribution for one of the test partitions, both for non-MDR (a) and MDR patients (b). For non-MDR patients, note that the output distribution is clearly shifted towards low probability values, as would be desirable. However, the lower number of MDR patients makes it di cult to characterize the output distribution for these patients. In particular, the mode of the distribution does not seem to be as clearly shifted to the right as would be desirable. A more detailed analysis, considering more MDR patients, could help to identify the reasons for this behaviour and therefore to propose strategies to modify the design of the classi er. 5
Multi-drug resistance is a growing and worldwide problem in current societies, directly associated with the use of antimicrobial drugs. The large-scale and sometimes inappropriate use of antibiotics in hospitals in general, and ICUs in particular, has accelerated the emergence of resistant bacteria. This means that bacteria that were sensitive to di erent antibiotics become resistant and, as a result, the length of stay, morbidity, mortality rates and healthcare costs may increase considerably. It has been estimated that antibiotic resistance would a ect to 10 million people by 2050, with a cost up to 100 trillion USD [oAR16].
These alarming gures have resulted in big e orts against bacterial pathogens. Recently, several studies have analyzed the DNA-sequencing using machine learning techniques to predict antimicrobial resistance [NLM+19]. Although the whole genome sequencing provides the potential to predict antimicrobial susceptibility, it is not available in all hospitals nowadays. To overcome this drawback, we analyze MTS recorded at the EHR of the ICU, speci cally, we focus on the use of antibiotics and mechanical ventilation.
Leveraging EHR data, the goal of this paper is to nd underlying and complex relationships between the use of antibiotics, mechanical ventilation and MDR by using LSTM networks. We evaluate the performance when di erent training strategies are considered to deal with imbalanced classes. The models presented in this work could speed up the work ow of the ICU, helping to identify and isolate patients potentially at risk of MDR.
This study was conducted using only two sources of MTS. To generalize the obtained conclusions, di erent MTS should be included, as well as clinical and demographic data such as age, gender or diagnoses. As future research, we plan to explore interpretable methods to nd the risk factors associated to multi-drug resistance. 5.0.1
This work has been partly supported by the Institute of Health Carlos III, Spain (grant DTS 17/00158), by the Spanish Ministry of Economy under the research project TEC2016-75361-R, by Project Ref. F656 nanced by Rey Juan Carlos University, by the Young Researchers R&D Project Ref. 2020-661, nanced by Rey Juan Carlos University and Community of Madrid (Spain), and by the Youth Employment Initiative (YEI) R&D Project Ref. TIC-11649 nanced by the Community of Madrid (Spain). [CBA15]
K. Caballero Barajas and R. Akella. Dynamically modeling patient's health state from electronic medical records: A time series approach. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 69{78, 2015. [DGK02] [Gra12]
A. Graves. Supervised sequence labelling with recurrent neural networks. Springer, 2012.
S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural computation, 9:1735{80, 1997.