=Paper=
{{Paper
|id=Vol-2820/SP4HC_paper2
|storemode=property
|title=Temporal Feature Selection for Characterizing
Antimicrobial Multidrug Resistance in the Intensive Care
Unit
|pdfUrl=https://ceur-ws.org/Vol-2820/AAI4H-11.pdf
|volume=Vol-2820
|authors=Óscar Escudero-Arnanz,Inmaculada Mora-Jiménez,Sergio Martínez-Agüero,Joaquín Álvarez-Rodríguez,Cristina Soguero-Ruiz
|dblpUrl=https://dblp.org/rec/conf/ecai/Escudero-Arnanz20
}}
==Temporal Feature Selection for Characterizing
Antimicrobial Multidrug Resistance in the Intensive Care
Unit==
https://ceur-ws.org/Vol-2820/AAI4H-11.pdf
Temporal Feature Selection for Characterizing
Antimicrobial Multidrug Resistance in the Intensive Care
Unit
Óscar Escudero-Arnanz1 and Inmaculada Mora-Jiménez2 and Sergio Martı́nez-Agüero3
and Joaquı́n Álvarez-Rodrı́guez4 and Cristina Soguero-Ruiz5
Abstract. The emergence and increase of antimicrobial multidrug port by the World Health Organisation (WHO), it is estimated an in-
resistance (AMR) is a demographic and economic problem for cur- crease in deaths by 2050 caused by antimicrobial resistance, mainly
rent health systems. AMR is particularly problematic in clinical units affecting countries such as Africa and Asia [1]. This is a growing
such as the intensive care unit (ICU), where the risk of infection is problem which needs to be alleviated to avoid the consequences
high, principally due to the extensive use of antimicrobials and inva- that this could cause. In addition to the demographic effects, the
sive devices. In this work, we propose the use of different temporal increase in antimicrobial multidrug resistance, this is the resistance
feature selection and classification approaches to ascertain the most of a single bacterium to more than one antibiotic, has a major eco-
informative features and extract knowledge for characterizing AMR nomic impact, resulting in loss to the world economy of approxi-
in the ICU. For this purpose, a set of demographic and temporal fea- mately 7% of the Gross Domestic Product by 2050 [13]. From an
tures such as antibiotics taken daily by the patient and the use of economic viewpoint, patients infected with antimicrobial resistant
mechanical ventilation are considered. According to the results ob- bacteria present a higher cost for the healthcare system in compar-
tained in this work, it could be concluded that temporal features such ison to patients who are susceptible to microbial infection [7]. This
as mecanic ventilation provide powerful insights to predict AMR in is caused by the increasing difficulty in treating resistant organisms,
ICU. making it necessary the breakthrough of new strategies to combat
antibiotic resistance. Previous studies have proposed initial analysis
based on machine learning models to determine the result (suscepti-
1 INTRODUCTION ble/resistance) of the antibiogram (a test to measure the in vitro ac-
The discovery of antibiotics and their subsequent use in the clinical tivity of an antibiotic against a given bacterium, which is previously
practice represented a great scientific advance, improving the treat- isolated in the culture [12]) or to predict the probability of acquiring
ment of infectious diseases and thus saving millions of lives [10]. a hospital-acquired infection (nosocomial infection), specifically in
However, the excessive and incorrect use of antibiotics is contribut- the ICU [15].
ing a downturn in their effectiveness against bacterial infections, Focusing on a hospital environment, antimicrobial resistance can
caused by mutations and the acquisition of genetic information from be acquired by any hospitalised patient, increasing the probability of
other germs [18]. This fact makes infection control difficult and in- acquisition for patients admitted to the Intensive Care Unit (ICU).
creases the morbidity and mortality of previously treatable infectious The main reasons are the use of invasive devices, the intensity of
diseases such as malaria or acute respiratory diseases [1]. treatment and its duration, the high risk of transmission and expo-
The impact of antimicrobial multidrug resistance (AMR) can sure to antibiotics. The ICU can be considered as the epicenter of
cause an economic burden in hospitals and in the healthcare sys- development of antimicrobial resistance due to the high rate of noso-
tems, whose real outcomes still remain unknown. Following the re- comial infections (20-30% of all ICU admissions) [4]. However,
the period just before the patient is admitted to the ICU is begin-
1 Department of Signal Theory and Communications, Telematics and ning to take great importance, caused by the increase in the num-
Computing Systems, Rey Juan Carlos University, Madrid 28943, Spain, ber of patients arriving in the ICU infected by multi-resistant micro-
o.escudero.2016@alumnos.urjc.es
2 Department of Signal Theory and Communications, Telematics and Com- organisms [19]. A culture is usually performed to assess bacteria sus-
puting Systems, Rey Juan Carlos University, Madrid 28943, Spain, inmac- ceptibility/resistance to series of antibiotics. Firstly, an organic sam-
ulada.mora@urjc.es ple from the patient (blood or urine samples, among others) is ob-
3 Department of Signal Theory and Communications, Telematics and Com-
tained which allows the study of the microorganisms present in their
puting Systems, Rey Juan Carlos University, Madrid 28943, Spain, ser-
gio.martinez@urjc.es system. Then, the antibiogram is carried out. The result of the an-
4 Intensive Care Department, University Hospital of Fuenlabrada, Madrid tibiogram represents the pair antibiotic/sensibility. Therefore, based
28942, Spain, joaquin.alvarez@salud.madrid.org on this results, we consider that patients did not acquired the multi-
5 Department of Signal Theory and Communications, Telematics and
resistant bacteria in the ICU if the culture’s result is positive within
Computing Systems, Rey Juan Carlos University, Madrid 28943, Spain,
cristina.soguero@urjc.es the first 48 hours of the patient’s admission, otherwise, the AMR oc-
Copyright © 2020 for this paper by its authors. Use permitted under Cre- cur during the ICU stay.
ative Commons License Attribution 4.0 International (CC BY 4.0). This The excessive use of antimicrobials during the stay of patients in
volume is published and copyrighted by its editors. Advances in Artificial the ICU (some studies corroborate that more than 60% of patients
Intelligence for Healthcare, September 4, 2020, Virtual Workshop.
take antibiotics during their ICU stay [4]) along with other factors 2.2 Imbalanced sampling
discussed above, facilitate the emergence of AMR, making this prob-
lem the target to be treated. We will study the daily use of antibiotics In healthcare-related data sets, it is very common to deal with im-
and mechanical ventilation (MV) in the ICU at University Hospital balanced data [8], i. e., one class predominates over the other. This
of Fuenlabrada, Madrid, Spain. The final aim consists in determining imbalance is a challenge for designing data-driven models, since con-
the risk factors that best characterize the evolution of critical patients ventional approaches will mostly learn from the majority class and
as well as the relevance to identify patients with AMR. To this end, lead to biased models, reducing the performance for the minority
we apply hypothesis tests, linear and non-linear learning algorithms. class. Data-driven approaches tend to learn better the mapping of
The rest of the paper is organized as follows. Section 2 introduces patients belonging to the majority class (far more numerous) than
the methods used for the temporal patient characterization. In Sec- that of the minority class. To tackle this challenge, several strategies
tion 3, a brief description of the data set is presented, while in Sec- could be followed [8]. In this work, we followed a random under-
tion 4 the experimental work and prediction results are shown. Fi- sampling strategy [20] with no replacement for the majority class.
nally, discussion and conclusions are presented in Section 5. The final sample size is such that the class frequency is similar. Thus,
the number of patients of the majority class is matched before train-
ing the model according to the number of patients of the minority
class. The undersampling process and subsequent model training is
2 METHODS repeated several times not to be conditioned to a particular subsam-
pling, providing statistics on the performance. We benchmark the re-
Notation sults obtained with random undersmpling with a synthetic minority
oversampling technique (SMOTE), which consists of oversampling
In this paper, each sample is a patient represented by a set of D fea- the examples in the minority class [6].
tures, being each feature composed by a time series of T consecutive
time slots. Therefore, the data associated to the i-th patient can be ar- 2.3 Classification Approaches
ranged in a feature matrix Xi = [x1i , x2i , . . . , xTi ] ∈ RD×T . Where
the column vector xti contains the D features of the i-th patient in Classification approaches encompasses statistical techniques to build
the time slot t. Thus, xti can be represented as the column vector models based on the underlying relationships among data. The set
xti = [xti,1 , xti,2 , · · · , xti,D ]T , where [.]T denotes the transpose op- of N available samples is split into two independent subsets, named
erator and xti,d shows the value of the d-th feature associated to the training set and test set. The former is used to create the classifier
i-th patient in the t-th time slot. Since we are tackling with a bi- following a learning process, whereas the latter is used to evaluate
nary classification task, we have considered the label ‘1’ to identify the performance of the built model. Normally, the 70% of samples
patients with AMR, and the label ‘0’ to identify patients with non- are randomly assigned to the training set and the rest to the test [5].
AMR. Therefore, the label (desired output) for the i-th patient is de-
fined by yi , whereas the output provided by the model is represented 2.3.1 Logistic Regression
as ŷi .
The model provided by Logistic Regression (LR) is a linear combi-
nation of the different features. Despite its name, it is a classification
approach since the result of the linear combination is the input to
2.1 Feature Selection a logistic function. To carry out the linear combination of the fea-
tures, a set of coefficients wi di=1 should be found by optimizing a
There are different methods for feature selection in the literature. binary cross-entropy cost function. In this work, we considered a
The goal is to eliminate features that may be noisy, irrelevant or re- regularized term in the cost function, in particular the Ridge regu-
dundant when building a data-driven model [17]. Also, selecting the larization [9] for preventing the model from overfitting. To find an
most important features can increase the knowledge and the model appropriate value for the hyperparameter weighting the penalization
interpretability. In this work, we want to select features based on hy- term in the cost function, named penalty coefficient C > 0, we fol-
pothesis tests. For each feature, our null hypothesis is that there is lowed a 5 fold cross-validation approach on the training set.
no difference between the two populations (AMR patients and non-
AMR patients). If there is no evidence to rule out the null hypothesis,
2.3.2 Decision Trees
then the tested feature is not selected. Since we are dealing with bi-
nary and numerical features, we evaluate a test of proportions for the Decision trees (DT) are non-parametric classifiers which can be
first kind of features, and a two-sample Kolmogorov-Smirnov test for graphically represented in a tree shape as a hierarchical structure
the latter. starting from a root node [14]. For building the tree, a recursive split-
Two-proportion z-test. This hypothesis test evaluates whether the ting process is carried out dividing the decision space into subspaces
presence on a single feature differs in two populations [16]. The null based on a criteria related to entropy or Gini index. In this work,
hypothesis states that there is no evidence of difference in the pro- we have chosen the Gini criterion to make the splitting process [11].
portion between both populations, whereas the opposite applies for When a node is created, a region in the feature space is splitted in two
the alternative hypothesis. parts. A label is assigned to each partition according to the majority
Two-sample Kolmogorov-Smirnov test. It is a hypothesis test based class among the training samples in that particular partition. One ad-
on the empirical distribution function and used to estimate whether vantage of DT is the model interpretability, that partly relies on the
values of the same feature in two populations are from the same con- fact that the most discriminative features are closest to the root node,
tinuous distribution [2]. An advantage of this test over parametric what implicitly could be considered as a feature selection process.
test is the independence of the statistic from the expected frequency In this work we considered DT built following the classification
distribution, depending only on the sample size. and regression tree algorithm named CART [3], since it has been
extensively used in the literature when dealing with heterogeneous we have considered the number of hours the patient was assisted with
features (numerical and categorical). mechanical ventilation. The use of these features is supported by the
fact that the incorrect and excessive use of antibiotics or external de-
vices are one of the main causes for the AMR onset. In addition, two
3 DATASET DESCRIPTION AND TEMPORAL demographic features (not time-dependent), the age and the gender
FEATURES of the patient, have been used as input of the models.
In this work, an anonymized dataset provided by the University Hos-
pital of Fuenlabrada (UHF) in Madrid (Spain) has been analysed.
This dataset contains demographic and clinical features of 2889 pa-
tients admitted in the ICU of the UHF during a period of 13 years,
from 2004 to 2016. The goal is leverage these data to character-
ize AMR in the ICU. From a clinical viewpoint, clinicians at UHF
considered that patients with a positive culture (presence of multi-
resistant germs) in the first 48 hours, had acquired the AMR before
their ICU admission. On the contrary, we considered that patients
with a positive culture after the early 48 hours of their admission,
had acquired the AMR during their ICU stay. Therefore, 507 of the
total number of patients acquired antimicrobial resistance, of which
171 (33.73%) acquired AMR before their ICU admission and 336
(66.27%) during their ICU stay. The average age of AMR patients
is 62.39 years, and 59.29 for non-AMR patients. In both cases, the
standard deviation is high (13.00 and 16.02, respectively). Regarding
gender, the percentage of men is higher for both AMR and non-AMR
patients (63.71% and 61.13%, respectively). Figure 1. Temporal feature matrix construction with a time window of 7
consecutive slots of 24 hours: AMR patient (upper panel) and non-AMR
The dataset has been preprocessed to characterize the evolution of
patient (bottom panel). For the AMR patient, t6 represents the time slot
the patient’s health status by a set of features suitable to feed the pre- closest to the date the positive culture is performed. For the non-AMR
dictive model inputs. Thus, the d-th temporal feature corresponding patient, t0 represents the time slot closest to the patient’s ICU admission.
to the i-th patient is represented by a a row vector associated to a T -
days time window, and it is given by: xi,d = [x1i,d , x2i,d , · · · , xTi,d ],
with d = 1, · · · , D. In this work, we have considered T = 7 time We present in Fig. 2 the percentage of AMR and non-AMR pa-
slots, i.e, the temporal characterization of a patient has been done tients who take each family of antibiotics. Note that this percentage
in a 7-days time window, with t0 the first 24 hours from the ICU is similar for some families of antibiotics such as Broad-Spectrum
admission for the non-AMR patients. Regarding AMR patients, the Penicillins, Quinolones and Lipopeptides. However, the percentage
time slot t0 represents the time slot furthest from the first positive of Antifungals, Glycopeptides and Carbapenemes is higher for AMR
culture, and therefore, closest to the ICU admission. Since the length patients, while non-AMR patients present a higher percentage of
of the ICU stay can be shorter than 7 days for some patients, we Penicillins, among others.
created a new binary feature, called mask, which takes a value of
‘1’ if the patient was in the ICU at this time slot, or ‘0’ otherwise.
The upper panel in Fig. 1 illustrates ficticious values for the mask
and the D features associated to one AMR patient. In this example,
since the culture flagged as positive the fifth day since the patient’s
ICU admission, all features assigned to t0 and t1 have null values.
The bottom panel in Fig. 1 represents the hypothetical values for the
mask and features associated to a potential non-AMR patient with a
stay of at least 7 days, being t0 the time slot nearest to the patient’s
ICU admission.
The features represented as xi,d in Fig. 1 are associated to the
family of antibiotics taken by the patient (23 features), as well
as to the mechanical ventilation (MV), to the result of the albu- Figure 2. Percentage of patients with respect to the total of each
min blood test and to the number of times this blood test was population (AMR and non-AMR) taking a particular family of antibiotics.
required. The families of the antibiotics the patient can take are
the following: Aminoglycosides (AMG), Antifungals (ATF), Car-
bapenemes (CAR), 1st generation Cephalosporins (CF1), 2nd gener-
ation Cephalosporins (CF2), 3rd generation Cephalosporins (CF3), 4 EXPERIMENTS AND RESULTS
4th generation Cephalosporins (CF4), unclassified antibiotics (Oth-
ers), Glycyclines (GCC),Glycopeptides (GLI), Lincosamides (LIN), The goal of this work was twofold. On the one hand, a feature se-
Lipopeptides (LIP), Macrolides (MAC), Monobactamas (MON), Ni- lection strategy was applied to find the most relevant features to dis-
troimidazolics (NTI), Miscellaneous (OTR), Oxazolidinones (OXA), criminate between AMR and non-AMR patients. On the other hand,
Broad-Spectrum Penicillins (PAP), Penicillins (PEN), Polypeptides the chosen features were considered to evaluate the potential of dif-
(POL), Quinolones (QUI), Sulfamides (SUL) and Tetracyclines ferent prediction models when classifying AMR and non-AMR pa-
(TTC). Regarding the feature associated to MV, for each time slot tients. Towards that end, we start this section by discussing the ex-
perimental set-up, then we present the feature selection process and test subsets of 1000 subsamplings are provided using random under-
the prediction results. sampling and SMOTE to balance the data. In general, the LR model
(a linear model) achieves better results, especially in terms of Sensi-
tivity (69.97 ± 3.68). On the contrary, better results in term of Speci-
4.1 Experimental Setup ficity (86.13 ± 1.87) are obtained when considering DT (non-linear
model). The results obtained through the use of SMOTE for LR im-
The methodology to select relevant features and train different clas-
prove, except Sensitivity. On the other hand, in DT, better results
sifiers is as follows. First, patients in the dataset were randomly sepa-
are obtained for Specificity and Accuracy, while the other metrics
rated, assigning the 70% to the train set and the 30% to the test set [5].
worsen.
In order to reduce the potential bias in the results produced by good
Figure 3 shows the importance of the features provided by 1000
or bad partitions, we repeat this process 1000 times. The metrics used
different models when considering LR and DT. To estimate the fea-
for measuring the performance of the classifiers are the mean and the
ture importance in LR, we have considered the absolute values of
standard deviation of the Accuracy, Specificity, Sensitivity, F1-score
the weights associated to the features, while we have used the Gini
and the Area Under the Curve (AUC).
index in DT. The results are presented in box-plots, with features
For tuning hyperparameters, a 5-fold cross-validation strategy was
sorted increasingly according to median of the p-values provided
considered in the training set. For the LR models the hyperparameter
by 1000 models. Features with the highest importance are approx-
used was the penalty coefficient C ∈ {0.001, 0.005, 0.01, 0.05, 0.1,
imately the same in both classifiers, highlighting MV in some time
0.5, 0.75, 1.0}. The hyperparameters associated to the decision tree
slots, the blood albumin value and the age of the patient.
were the depth of the tree (ranging from 4 to 22) and the minimum
of samples per leaf (between 6 and 15).
0.8
0.7
4.2 Temporal Feature Selection 0.6
0.5
We performed a hypothesis test for each time slot and for all features
0.4
described in Section 3, except for demographic features due to the
0.3
non-dependence in time of this kind of features. For both imbalanced
0.2
and balanced data, we considered the p-value provided by the two-
0.1
proportion z-test for antibiotics and by the two-sample Kolmogorov-
Smirnov test for MV and albumin (see Table 1). In the case of imbal- 0.0
Albumin(Value)-t6
MV(hours)-t6
Albumin(Count)-t0
Albumin(Value)-t4
MV(hours)-t5
Albumin(Value)-t0
Albumin(Value)-t3
OXA-t6
Age
GLI-t0
OXA-t0
Albumin(Value)-t5
PEN-t5
ATF-t2
MV(hours)-t4
Albumin(Count)-t1
AMG-t1
PEN-t1
Albumin(Value)-t2
AMG-t0
PEN-t0
PEN-t3
PEN-t4
ATF-t0
Others-t2
ATF-t1
CF3-t0
ATF-t3
QUI-t1
OXA-t4
Others-t6
ATF-t5
MV(hours)-t3
QUI-t0
Albumin(Value)-t1
NTI-t0
PEN-t2
GLI-t1
Others-t4
OXA-t1
Others-t3
Others-t5
NTI-t1
MV(hours)-t2
CF3-t1
OXA-t5
PEN-t6
ATF-t6
ATF-t4
OXA-t2
Gender
OXA-t3
anced data, we determined as significant features those with a p-value
< 0.1. When considering balanced datasets, we perform N = 1000
subsamplings of the majority class and obtain the median of the p-
values, selecting those features such that the median of the p-values
is lower than 0.1. (a)
To perform the experiments, we have used those features selected
by the above tests when using balanced subsets, together with the
0.4
demographic features of the patient. We have selected those fea-
tures that are statistically significant (p-value ¡ 0.1) during the first
0.3
48 hours (t0 and t1 ), from 48 hours (t2 ,t3 ,t4 ,t5 , and t6 ) onwards or
throughout the time window (from t0 to t6 ). According to these con- 0.2
ditions, we have obtained the following features: all time slots of
ATF, PEN, OXA, and Albumin (Value), from time slot t2 to t6 for 0.1
Others and MV (hours), and the first two time slots for AMG, CF3,
GLI, NTI, QUI and Albumin (Count). Some of these features are 0.0
MV(hours)-t6
Albumin(Value)-t6
Albumin(Value)-t0
Albumin(Value)-t4
Albumin(Value)-t5
Age
Albumin(Count)-t0
Albumin(Value)-t3
Albumin(Value)-t1
Others-t6
Others-t3
Others-t4
Others-t5
Gender
GLI-t0
CF3-t1
GLI-t1
NTI-t0
Others-t2
ATF-t4
CF3-t0
ATF-t6
ATF-t5
OXA-t0
ATF-t3
ATF-t2
ATF-t1
ATF-t0
AMG-t1
NTI-t1
OXA-t5
OXA-t1
PEN-t6
Albumin(Count)-t1
Albumin(Value)-t2
MV(hours)-t5
MV(hours)-t4
MV(hours)-t3
MV(hours)-t2
QUI-t1
QUI-t0
PEN-t5
OXA-t2
PEN-t4
PEN-t3
PEN-t2
PEN-t1
PEN-t0
OXA-t6
OXA-t4
OXA-t3
AMG-t0
clinically relevant. For example, QUI and AMG are antimicrobial
families employed to treat the pseudomona aeruginosa infections,
OXA family are the main antimicrobial given to tackle the staphy-
lococcus aureus (both pseudomonas aeruginosa and staphylococcus
aureus are the most common MDR bacteria). The mechanical ven- (b)
tilation and the level of albumin in the blood are related to the pa-
tient’s state of health. The p-values associated to these features and
time slots are in bold in Table 1. Figure 3. Box-plots of the importance of features provided by 1000
different models: (a) absolute value of the coefficients for the LR models; (b)
importance based on gini index.
4.3 Prediction Results
In this subsection, the results of predicting whether a patient will
be considered AMR or non-AMR are presented in Table 2. For the
prediction, we considered both a linear (LR) and non-linear (DT) 5 CONCLUSIONS
models, designed using the features selected in Subsection 4.2.
Several conclusions can be obtained from Table 2, where the mean Nowadays, AMR has become a real and growing problem due to
and standard deviation of several performance measurements on the the inappropriate use of antimicrobials. Bacteria that were previously
Table 1. p-value obtained when performing a hypothesis test on a single feature per time slot, associated to AMR and non-AMR populations. First p-value in
the cell corresponds to unbalanced datasets, while the second value shows the median of the p-values for balanced dataset on 1000 subsamplings. Bold figures
denote those features satisfying the alternative hypothesis (p-value lower than 0.1).
Time Slot
t0 t1 t2 t3 t4 t5 t6
Feature
2.943e-03 6.167e-03 1.606e-02 5.128e-02 1.467e-01 1.929e-01 8.895e-01
AMG
7.440e-03 1.411e-02 3.733e-02 9.571e-02 2.465e-01 2.781e-01 7.193e-01
4.497e-07 1.483e-07 4.722e-09 1.543e-07 8.290e-08 2.623e-06 6.851e-03
ATF
4.948e-04 6.484e-04 1.938e-04 5.590e-04 3.763e-04 1.656e-03 4.029e-02
1.186e-01 7.480e-02 1.003e-01 2.500e-03 5.866e-03 2.408e-04 1.924e-04
CAR
2.130e-01 1.958e-01 2.171e-01 6.678e-02 5.954e-02 2.019e-02 5.289e-03
2.808e-02 2.199e-01 2.432e-01 2.430e-01 8.337e-02 4.779e-01 7.706e-01
CF1
3.229e-02 3.127e-01 3.132e-01 2.541e-01 5.744e-02 5.242e-01 7.044e-01
4.798e-01 1.0e+00 1.0e+00 6.686e-01 1.405e-01 3.281e-01 6.444e-01
CF2
1.0e+00 1.0E+00 1.0e+00 1.0e+00 3.169e-01 3.169e-01 1.0e+00
1.230e-03 1.608e-02 3.104e-02 9.467e-02 6.654e-01 8.983e-01 1.845e-01
CF3
4.685e-03 3.716e-02 5.323e-02 1.321e-01 6.257e-01 6.527e-01 3.128e-01
2.870e-01 3.282e-01 3.930e-01 3.498e-01 3.258e-02 5.092e-02 2.525e-01
CF4
5.024e-01 5.032e-01 5.669e-01 4.227e-01 1.978e-01 1.386e-01 3.330e-01
6.868e-02 1.405e-01 4.077e-04 3.253e-04 4.630e-05 1.042e-04 2.756e-02
Others
2.079e-01 2.723e-01 1.712e-02 1.133e-02 5.216e-03 7.956e-03 9.556e-02
3.866e-01 5.120e-01 6.722e-01 5.981e-01 5.222e-01 6.900e-01 2.274e-01
GCC
3.167e-01 3.168e-01 3.168e-01 3.169e-01 3.169e-01 3.169e-01 3.170e-01
1.406e-04 4.152e-03 4.495e-02 2.368e-02 2.255e-04 8.859e-06 8.654e-06
GLI
5.365e-03 3.866e-02 1.243e-01 1.008e-01 8.780e-03 3.000e-03 1.531e-03
1.288e-01 5.057e-01 7.211e-01 8.0236e-01 7.746e-01 6.766e-01 7.989e-01
LIN
2.007e-01 3.989e-01 6.486e-01 6.322e-01 6.326e-01 6.331e-01 6.343e-01
6.175e-01 2.537e-01 6.501e-01 1.768e-01 3.314e-01 2.781e-03 6.444e-01
LIP
1.0e+00 3.168e-01 1.0e+00 3.169e-01 3.169e-01 8.263e-02 1.0e+00
5.310e-01 7.344e-01 6.374e-01 8.605e-01 3.339e-01 1.379e-02 9.732e-02
MAC
5.223e-01 7.034e-01 7.036e-01 7.038e-01 5.239e-01 8.075e-02 2.454e-01
1.852e-01 6.557e-01 4.626e-01 5.367e-01 4.164e-01 4.321e-02 1.021e-02
MON
1.563e-01 5.624e-01 6.532e-01 6.533e-01 6.534e-01 2.545e-01 9.410e-02
1.642e-02 3.691e-02 3.441e-01 5.944e-01 8.174e-01 1.247e-01 5.465e-01
NTI
4.325e-02 6.454e-02 4.214e-01 6.640e-01 7.086e-01 2.253e-01 5.842e-01
1.0e+00 1.0e+00 6.501e-01 6.686e-01 1.405e-01 3.281e-01 2.524e-02
OTR
1.0e+00 1.0e+00 1.0e+00 1.0e+00 3.169e-01 3.169e-01 1.568e-01
2.821e-04 7.593e-03 8.882e-03 3.699e-03 1.195e-05 3.403e-04 1.686e-05
OXA
9.830e-03 6.735e-02 7.648e-02 4.899e-02 6.101e-03 2.435e-02 3.685e-03
1.319e-01 6.248e-02 1.286e-02 1.798e-02 5.152e-02 1.540e-01 8.518e-01
PAP
2.256e-01 1.048e-01 3.750e-02 6.685e-02 1.387e-01 2.650e-01 6.804e-01
4.679e-06 9.010e-06 2.468e-04 1.372e-04 1.342e-03 3.036e-03 1.492e-02
PEN
1.030e-05 1.966e-05 1.313e-03 5.027e-04 4.512e-03 1.231e-02 4.455e-02
6.175e-01 2.537e-01 2.034e-03 9.855e-04 1.062e-02 7.761e-03 1.985e-07
POL
1.0e+00 3.168e-01 8.241e-02 8.250e-02 8.256e-02 1.782e-01 4.516e-03
9.795e-03 1.982e-02 4.590e-03 4.443e-02 7.272e-01 7.191e-01 8.757e-01
QUI
3.028e-02 5.761e-02 1.700e-02 7.849e-02 5.774e-01 6.558e-01 6.735e-01
8.343e-01 6.689e-01 1.280e-01 2.232e-01 4.833e-02 3.959e-03 4.582e-01
SUL
7.032e-01 4.758e-01 2.429e-01 4.003e-01 3.113e-01 8.586e-02 5.904e-01
4.105e-01 1.686e-01 3.659e-01 8.036e-01 8.793e-01 3.018e-01 4.320e-01
TTC
5.623e-01 3.153e-01 4.762e-01 5.627e-01 5.628e-01 4.123e-01 4.128e-01
7.451e-01 4.393e-01 1.441e-04 2.180e-12 0.0e+00 1.882e-26 1.332e-15
MV (hours)
7.705e-01 7.423e-01 3.997e-03 1.238e-07 9.859e-14 5.943e-16 1.046e-19
5.551e-16 4.593e-06 1.646e-04 9.611e-05 1.145e-03 2.981e-01 1.332e-15
Albumin (Value)
2.016e-19 6.524e-04 5.390e-03 4.703e-03 2.541e-02 5.892e-01 2.689e-10
5.551e-16 6.335e-06 1.646e-04 3.506e-04 4.085e-02 6.951e-01 1.665e-15
Albumin (Count)
2.016e-19 9.377e-04 7.237e-03 1.065e-02 2.209e-01 9.211e-01 6.278e-10
Table 2. Mean ± standard deviation of several performance measurements (Specificity, Sensitivity, Accuracy, F1-score and AUC) on 1000 test sets when
designing a lineal model (LR) and a non linear model (DT).
Training Strat. Model Specificity Sensitivity Accuracy F1-score AUC
LR 73.45 ± 2.09 69.97 ± 3.68 72.83 ± 1.64 47.79 ± 2.54 71.71 ± 1.82
Random Under.
DT 78.52 ± 3.77 60.45 ± 4.38 75.23 ± 2.84 47.08 ± 3.25 69.48 ± 2.19
LR 77.52 ± 1.78 66.6 ± 3.91 75.58 ± 1.44 49.2 ± 2.62 72.06 ± 1.9
SMOTE
DT 86.34 ± 1.87 48.78 ± 4.63 79.65 ± 1.56 45.97 ± 3.5 67.56 ± 2.25
easily treatable have now become an issue difficult to deal with, es- [6] Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip
pecially in the ICUs. In these units, AMR has created a great impact Kegelmeyer, ‘Smote: synthetic minority over-sampling technique’,
Journal of artificial intelligence research, 16, 321–357, (2002).
on morbidity, hospital costs, and sometimes patient survival. [7] Sara E Cosgrove, ‘The relationship between antimicrobial resistance
It is necessary to be aware of the growing problem caused by the and patient outcomes: mortality, length of hospital stay, and health
expansion of AMR, for which new research, efforts, and approaches care costs’, Clinical Infectious Diseases, 42(Supplement 2), S82–S89,
are needed to prevent further spread of AMR. The use of automatic (2006).
learning methods is a very useful tool to solve problems related to the [8] José F Dı́ez-Pastor, Juan J Rodrı́guez, César Garcı́a-Osorio, and Lud-
mila I Kuncheva, ‘Random balance: ensembles of variable priors clas-
clinical environment following a data-driven strategy. These methods sifiers for imbalanced data’, Knowledge-Based Systems, 85, 96–111,
allows us to reduce the time of detection of infectious diseases, re- (2015).
sulting in a reduction in the number of deaths as well as in health [9] Jianqing Fan and Runze Li, ‘Statistical challenges with high dimen-
economic costs. sionality: Feature selection in knowledge discovery’, arXiv preprint
math/0602133, (2006).
In this work, we proposed the use of feature selection and machine [10] Infectious Diseases Society of America (IDSA), ‘Combating antimi-
learning approaches to extract knowledge and predict the appearance crobial resistance: policy recommendations to save lives’, Clinical In-
of AMR of patients admitted in the ICU. Features such as the per- fectious Diseases, 52(suppl 5), S397–S428, (2011).
formance provided by LR (71.71% AUC) suggests that the analysis [11] G James, D Witten, T Hastie, and R Tibshirani, An Introduction to Sta-
presented in this paper could be a first step to identify the bacteria tistical Learning with Applications in R, Springer, 2013.
[12] Sergio Martı́nez-Agüero, Inmaculada Mora-Jiménez, Jon Lérida-
appearance and isolate the patients at risk of AMR. Garcı́a, Joaquı́n Álvarez-Rodrı́guez, and Cristina Soguero-Ruiz, ‘Ma-
As future work, we propose the analysis of more features related to chine learning techniques to identify antimicrobial resistance in the in-
the patients such as blood samples or vital signs, as wells as the use tensive care unit’, Entropy, 21(6), 603, (2019).
of more advanced machine learning methods, as for example, long [13] Marc Mendelson and Malebona Precious Matsoso, ‘The world health
organization global action plan for antimicrobial resistance’, SAMJ:
short-term memory networks which are capable of learning long- South African Medical Journal, 105(5), 325–325, (2015).
term dependencies. [14] J. Ross Quinlan, ‘Induction of decision trees’, Machine learning, 1(1),
81–106, (1986).
[15] Paz Revuelta-Zamorano, Alberto Sánchez, José Luis Rojo-Álvarez,
ACKNOWLEDGEMENTS Joaquı́n Álvarez-Rodrı́guez, Javier Ramos-López, and Cristina
This work has been partly supported by the Institute of Health Car- Soguero-Ruiz, ‘Prediction of healthcare associated infections in an in-
tensive care unit using machine learning and big data tools’, in XIV
los III, Spain (grant DTS 17/00158), by the Spanish Ministry of Mediterranean Conference on Medical and Biological Engineering and
Economy, Industry and Competitiveness under the Research Project Computing 2016, pp. 840–845. Springer, (2016).
Klinilycs (TEC2016-75361-R), by the Science and Innovation Min- [16] Pilar Talón-Ballestero, Lydia González-Serrano, Cristina Soguero-
istry Grants AAVis-BMR (PID2019-107768RA-I00) and BigTheory Ruiz, Sergio Muñoz-Romero, and José Luis Rojo-Álvarez, ‘Using big
(PID2019-106623RB-C41), by Project Ref. F656 financed by Rey data from customer relationship management information systems to
determine the client profile in the hotel sector’, Tourism Management,
Juan Carlos University, by the Young Researchers R&D Project Ref. 68, 187–197, (2018).
2020-661, financed by Rey Juan Carlos University and Community [17] Jiliang Tang, Salem Alelyani, and Huan Liu, ‘Feature selection for clas-
of Madrid (Spain), and by the Youth Employment Initiative (YEI) sification: A review’, Data classification: Algorithms and applications,
R&D Project Ref. TIC-11649 financed by the Community of Madrid 37, (2014).
[18] Magnus Unemo and William M Shafer, ‘Antimicrobial resistance in
(Spain). neisseria gonorrhoeae in the 21st century: past, evolution, and future’,
Clinical microbiology reviews, 27(3), 587–613, (2014).
REFERENCES [19] Dirk Vogelaers, De Bels, et al., ‘Patterns of antimicrobial therapy in
severe nosocomial infections: empiric choices, proportion of appropri-
[1] Matteo Bassetti, Garyphallia Poulakou, Etienne Ruppe, Emilio Bouza, ate therapy, and adaptation rates—a multicentre, observational survey
Sebastian J Van Hal, and Adrian Brink, ‘Antimicrobial resistance in in critically ill patients’, International journal of antimicrobial agents,
the next 30 years, humankind, bugs and drugs: a visionary approach’, 35(4), 375–381, (2010).
Intensive care medicine, 43(10), 1464–1475, (2017). [20] Show-Jane Yen and Yue-Shi Lee, ‘Under-sampling approaches for im-
[2] W Baumgartner, P Weiß, and H Schindler, ‘A nonparametric test for the proving prediction of the minority class in an imbalanced dataset’, in
general two-sample problem’, Biometrics, 1129–1135, (1998). Intelligent Control and Automation, 731–740, Springer, (2006).
[3] L Breiman, JH Friedman, RA Olshen, and CJ Stone, Classification and
Regression Trees, Chapman and Hall, 1984.
[4] Nele Brusselaers, Dirk Vogelaers, and Stijn Blot, ‘The rising problem of
antimicrobial resistance in the intensive care unit’, Annals of intensive
care, 1(1), 47, (2011).
[5] Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and
Noemie Elhadad, ‘Intelligible models for healthcare: Predicting pneu-
monia risk and hospital 30-day readmission’, in Proceedings of the 21th
ACM SIGKDD international conference on knowledge discovery and
data mining, pp. 1721–1730, (2015).