Preventive, Predictive, Personalised and Participative (P4) medicine has the potential to not only vastly improve people's quality of life, but also to significantly reduce healthcare costs and improve its eficiency. Our research focuses on age-related diseases and explores the opportunities ofered by a data-driven approach to predict wellness states of ageing individuals, in contrast to the commonly adopted knowledge-driven approach that relies on easy-to-interpret metrics manually introduced by clinical experts. This is done by means of machine learning models applied on the My Smart Age with HIV (MySAwH) dataset, which is collected through a relatively new approach especially for older HIV patient cohorts. This includes Patient Related Outcomes values from mobile smartphone apps and activity traces from commercial-grade activity loggers. Our results show better predictive performance for the data-driven approach. We also show that a post hoc interpretation method applied to the predictive models can provide intelligible explanations that enable new forms of personalised and preventive medicine.
Medical practice is evolving rapidly, away from the traditional but
ineficient detect-and-cure approach, and towards a Preventive,
Predictive, Personalised and Participative (P4) vision that focuses
on extending people’s wellness state, with particular focus on
ageing individuals [
Our research explores specific opportunities ofered by
datadriven approaches to predictive care, in contrast to traditional,
knowledge-driven approaches that rely purely on clinical
expertise. Our focus is on age-related diseases, an emerging issue
for health care systems. The World Health Organisation
estimates that the proportion of people over 60 years of age will
reach 2 billion by 2050 [
We focus on two easy-to-interpret metrics that clinical
researchers have proposed to succinctly express the health status
of patients at a given point in time. The first is a measure of
frailty, designed to quantify the reduction of homeostatic reserves
available to an individual. High frailty is indicative of higher
risk of negative health outcomes, but it is also a potentially
reversible condition [
The second measure of health considered in this work directly
reflects the more positive notion of healthy ageing [
As suggested by Belloni and Cesari [
Key to a successful operational definition of IC is the choice
of the variables used to characterise healthy ageing. [
Similarly to FI, ICI is computed from a subset of the available
variables, by manually selecting a cutof point for each variable
and simply counting, for a given patient, the variables with value
higher than the cutof for that patient. We have referred to this
approach that represents the common practice to assess health
condition in geriatric medicine as “knowledge-driven” (KD for
short), as it relies on easy-to-interpret metrics where the choice
of variables and of their cutof points is defined manually by
clinical experts. The usefulness of the ICI is demonstrated in [
In contrast to the dominant KD methods as described, in this paper we explore a complementary data-driven approach (DD) to predicting wellness states for long-term patients, using a combination of clinical, self-monitoring, and PRO (self-reporting) longitudinal observations that refer to the five IC domains. As we will see, while this approach removes the need for the clinical experts to directly define metrics such as the ICI, it also empowers them by deploying machine learning techniques that make the predictions easily interpretable.
Specifically, we focus on three dependent variables to characterise the “wellness state” of ageing individuals, namely, (i) Falls, indicating whether or not a person has experienced any falls within a given time period; (ii) SPPB, or Short Physical Performance Battery, measuring movement of the lower limbs, and (iii) Quality of Life (QoL). Using the MySAwH dataset for training, we can directly contrast the DD and KD approaches, and we show that we can achieve better predictive power without the need to rely on manually formulated ICI metrics. Furthermore, we also show that the models’ performance increases if we also include a single Frailty Index value for each patient, representing a “baseline” clinical assessment, in addition to the PRO and activity variables.
Finally, we combine well-established machine learning models with a post-hoc interpretation method and we show that satisfactory model performance can be achieved together with intelligible explanations, which provide the additional benefit of ranking the variables with respect to a prediction. Indeed, we show that the relative importance of the variables difers for each patient, indicating the important role played by intelligible models towards personalising healthcare. This capability also provides critical feedback to the clinicians, who can use this information in combination with their expertise to moderate the model’s predictions.
The idea of exploiting machine learning toward a general “health”
focus is certainly gaining popularity, also thanks to the
possibility of continuously monitoring well-being through wearable
activity trackers’ data. For instance, passive sensing techniques
have been recently exploited to assess both mental and physical
health [
A critical aspect for the successful application of ML to
medicine is the recent increased emphasis on the need for
explanations of ML systems [
The experimental dataset used in this paper was obtained from
the My Smart Age with HIV (MySAwH) [
Thus, the resulting dataset is highly heterogeneous as to the
data type, the geographical origin of patients and the acquisition
rate. It provides a comprehensive characterization of patients
from the broad determinants of health that impact aging,
complemented by those more specific to OPLWH, namely:
Activity tracking variables: Step count, sleep hours and
calories, which are collected daily using a
commercialgrade wearable activity tracker;
PRO variables: 56 categorical questions exploring
functional abilities and Quality of life (QoL). These are
collected monthly using a dedicated smartphone app;
Clinical variables: Comprehensive geriatric assessment
and HIV-specific variables, which are collected by
healthcare workers during study visits at time 0, 9 and 18 months.
37 of these variables were used to measure the Frailty
Index (FI) as defined in [
A preliminary analysis [
Outcomes. In this work, we focus on the task of predicting significant healty ageing indicators and QoL is one of such indicators. In addition to QoL, two other indicators were selected: Falls, that is an adverse outcome included among the geriatric syndroms, and the Short Physical Performance Battery (SPPB), a group of measures about movement ability with lower limbs (Guralnik et al., 2000) that can aid in the monitoring of function in older people. These three outcomes are chosen because they widely cover all 5 domains of IC. In summary, these are as follows (Fig. 1 shows their distributions):
Quality of Life (QoL): assessed using the EQ-5D5L standard, with values between 0 and 1; Falls: A binary outcome that evaluates to True if a patient has fallen at least once since the previous visit, and False otherwise; SPPB Index: a discrete index that assumes integer values between 0 and 12.
Observational data and feature space. The observational dataset used to predict these outcomes covers 18 months and is broken down into two time windows, reflecting the clinical assessment schedule (months 9 and 18), when the selected clinical outcomes are assessed. For each time window, we draw two sets of samples from the related observations, which we are going to use as ground truth to train our models.
The first consists of the patient-centric longitudinal data, including the activity tracking and the PRO variables. To this end, we further aggregated the resulting PRO time series and the activity tracking time series at regular intervals of 1 month, resulting in two sets of data points, one for each of the two windows. The second sample set augments the first with the FI values computed from the clinical variables measured during hospital visits at the beginning of each window, namely at times 0 and 9. This added value is a physician’s assessment that complements the patient-centric data point and can be interpreted as the baseline of the time series the data point comes from.
Formally, for each outcome o ∈ {QoL, SPP B, F alls } we write
Sampleo to denote the sample set containing the monthly samples
per patient. For each patient p, we denote a single sample in the
set Sampleo at month m = i + (j − 1) ∗ 9, corresponding to the
i-th observation, i ∈ [
The second sample set, denoted as SampleoF I , is built by adding p to each feature vector xi,j ∈ Sampleo , the FI value computed from the clinical variables measured during the hospital visit at the beginning of period j, i.e. at month 0 when j = 1 and 9 when j = 2.
Quality Assurance. Lastly, we performed a quality assurance step on the resulting sample sets. Within each time window, observations of PRO variables are sometimes incomplete, resulting in sequence gaps. The size of the gaps is 5 consecutive missing observations on average, with a max of 17, and we found 108 gaps per patient on average, with a max of 284 gaps (regardless of size). We performed imputation by interpolating missing data points in the time series, with an aim to achieve a balance between the size of the gaps and the performance of the predictive model. Clearly, interpolating very large gaps produces spurious data in the training set. We experimentally determined the max size of gaps that could be safely interpolated (five missing steps), by assessing the predictive performance of each of the models resulting from training sets obtained from more or less “aggressive” interpolation. After adjusting for missing data, the final training set contains 2,250 data points, with an average of 8 per patient. The construction of the dataset results in at most 16 samples per patient, for a total of 4,176 records, considering each month for each patient. 4
The objective of the data-driven approach is to predict each outcome at the time of a visit using the samples referring to the time window before that visit, as shown in Fig. 2.
To this end, the data-driven learning framework we built is depicted on the left side of Fig. 3. Each outcome o is predicted by two learning models Mo and MoF I , one for each of the two sample sets Sampleo and SampleoF I , respectively. We trained the two models separately and assessed the performance using standard KFold cross-validation (CV) on an 80% of the samples (χt r ain ) and a test phase on the remaining 20% samples (χt est ) of the corresponding sample set.
The DD approach is compared to the KD approach that represents the common practice in geriatric medicine. To this end, we built the KD learning framework depicted on the right hand side of Fig. 3. The approach aims at computing an ICI scores for each observation by manually selecting a subset V = {V1 . . . Vn } ⊆ V of the set of PRO and activity tracking variables V, specifying functions si (x ) to map each value x for variable Vi ∈ V to a score, and finally combining the individual scores si (x ) into a unique value. The variables are chosen to represent each of the five IC domains, namely locomotion, cognition, psychological, vitality and sensory capacity. For most of the variables Vi , a binary score is defined, i.e., si (x ) ∈ {0, 1}, based on a single threshold, for instance when Vi = stress level (from 1 to 10) the score is mapped 3 2 5 4 7 6 8 7 9 8 0 1 9 1 1 0 1 2 1 1 1 (a) QoL distribution (b) SPPB distribution
()
Data-Driven (DD) Approach Training in CV Regression Regression Classification MQoL_DD MSPPB_DD MFalls_DD
to 1 if the value is lower than 3 and 0 otherwise. Other variables
are mapped to a score in the [
Then, given a feature vector xip,j for patient p, the correspondp ing ICI value ICI (i, j, p) is computed as the sum of the si (xi,j [Vi ]) scores, normalised by the number of variables: p Íi|V:1 | si (xi,j [Vi ]) ICI (i, j, p) = n
Such an index is subject to an inevitable bias: the imposition of the physician’s interpretation on the choice of the variables of the subset, as well as on the thresholds and the arithmetic formula to be used.
Also for this case, a dataset consisting exclusively of the ICIs, SampleoIC I , and one consisting of the ICIs and the FI at the most recent visit, SampleoIC I ,F I , was isolated. This parallelism with what was done in the data-driven approach allowed to train 6 learning models with the datasets just described (M oIC I and M oIC I ,F I for each of the three outcomes o) and to compare the predictive performances between the two diferent approaches.
In this section we first discuss the performance of the predictive models, and then describe in detail our approach and results regarding model interpretation, which is reported as output to the DD approach in the left side of Fig. 3. The latter is a fundamental requirement in medicine, where the ability to provide medical doctors with an easy-to-understand interpretation of the model predictions is fundamental. This not only conveys confidence in the predictions, but also helps to make them actionable, i.e., in the form of recommendations to patients. We present examples of such interpretations, and their practical relevance, in Sec. 5.2.
The Gradient Boosting algorithm [
False
True (c) Falls distribution
Knowledge-Driven (KD) Approach Performance Comparison
QoL
(MAE)
(Acc, Prec, Rec)
Training in CV MQoL_KD MSPPB_KD MFalls_KD
Regression
Regression
Classification
performance from model interpretability would better suit our
needs. Thus, our results are based on a combination of
Gradient Boosting (the XGBoost implementation [
In our evaluation we compare our DD approach, illustrated in the left side of Fig. 3, with the KD approach based on a regression on the IC index (right hand side in the Figure). Fig. 4 shows the predictive performance of the models we tested, namely: the DD models trained with and without using FI as a feature, earlier referred to as MoF I and Mo , respectively; and the KD models, where again the expert may or may not consider FI. Results are presented using 1-MAPE (Mean Average Percentage Error) for the numerical outcomes QoL and SPPB on the left of the figure, and accuracy, precision, recall and F1 for Falls, on the right.
The results indicate a higher than 90% 1-MAPE for all cases in QoL and SPPB, while classification accuracy for Falls is higher than 84%. Further, the DD approach performs generally better than KD, and both benefit from using FI, with performance reaching 94.3%, 94.9% and 95% for QoL, SPPB, and Falls, respectively.
To note, in one case the KD approach returns a very low Recall when FI is not used. This can be explained by the strong imbalance of the majority “False” class (no Falls) relative to the small minority “True”.
The training sets used to generate these models combine patients from all three clinics. To account for possible diferences in data collection protocols between the clinics, we also created one separate model for each. The corresponding results are presented in Table 1 and are consistent with those presented above. Some anomalies appear in the Hong Kong models, and these are probably due to the small size of the training set.
Finally, Fig. 5 shows the MAE distribution grouped per clinical center for QoL and SPPB. This helps understanding the robustness of the models and to identify any non-homogeneity in the data. In particular, Hong Kong exhibits a higher number of outliers compared to Modena and Sydney, probably because of the small number of cases (33, compared to 128 in Modena and 100 in Sydney), which are also more homogeneous. These results suggest that developing separate models by stratifying across clinics and data collection centres may be beneficial for future, larger scale studies.
SHAP [
In our medical setting, this means that for each patient, in addition to the predicted outcome the clinician also receives a list of features, ranked in order of their relative importance in achieving the prediction. Importantly, these orders may difer for any two patients. This means that, using SHAP, we enable forms of personalised medicine whereby similar outcomes are explained in terms of diferent behaviour, represented for instance by diferent EMA features. An example appears in Fig. 6, showing two diferent sets of positively contributing (green) and negatively contributing (red) features for two patients with the same SPPB index (note that, for SPPB, higher is better, as this indicates the patient’s capacity of physical movement. In the case of Falls, for instance, the opposite would be true). Clearly, this added information may lead to diferent interventions for these two patients.
At the same time, SHAP provides global explanations, which characterise the contribution of each feature as a function of its range of values. For instance, Fig. 7 shows how the SV, indicating the overall contribution of one of these features (a PRO question), goes from negative to positive depending on the patients’ responses to this question, with a definite threshold of ⩾ 3.
We note that this capability essentially mimics the KD approach in that it identifies thresholds for the variables. While these are similar to the manually selected cutofs, in our DD approach these are automatically identified from the data, in a principled way. In the future, this explanation capability may underpin epidemiological studies where the precise characterisation of a populations of individuals enables new forms of preventive medicine.
In this paper we have proposed a novel, data-driven approach towards the definition of Intrinsic Capacity, aimed at quantifying and predicting the wellness state of old people who live with HIV. Using a cohort from a multi-centre prospective study as training set, we have shown that a machine learning model that predicts three specific wellness metrics (Falls, SPPB, and Quality of Life), performs equally or better than a manually-defined Intrinsic Capacity Index. At the same time, the model is interpretable, making it an ideal complement to expert-based assessment of wellness. w/o FI w/ FI w/o FI w/ FI w/o FI w/ FI 86% 84%
DD 100% 93% 94% 96% 89% 96% KD 0% 33% 0% 53% 38% 69%
DD 0% 33% 41% 68% 57% 68%