Deep learning models have achieved the start-of-the-art performance in survival analysis as they can handle censoring while learning complex nonlinear hidden representations directly from the raw data. However, the covariate effects on survival probabilities are difficult to explain using deep learning models. To address this challenge, we propose PseudoNAM - an interpretable model which uses pseudo values to efficiently handle censoring and uses neural additive networks to capture the nonlinearity in the covariates of the survival data. In particular, PseudoNAM uses neural additive models to jointly learn a linear combination of neural networks corresponding to each covariate and identifies the effect of the individual covariate on the output, and thus, is inherently interpretable. We show that our PseudoNAM outputs can be used in other survival models such as random survival forests to obtain improved survival prediction performance. Our experiments on three real-world survival analysis datasets demonstrate that our proposed models achieve similar or better performance (in terms of C-index and Brier scores) than the state-of-the-art survival methods. We showcase that PseudoNAM provides overall feature importance scores and feature-level interpretations (covariate effect on survival risk) for survival predictions at different time points.
Survival analysis
To address the limitations of the existing methods, we
propose a pseudo value based neural additive model, called
PseudoNAM, which directly models the complex
nonlinear time-varying effect of the covariate on the survival
function. Our PseudoNAM uses neural additive models
(NAM)
Our experiments on three real-world datasets demonstrate that PseudoNAM performs similar or better than the stateof-the-art survival analysis models while providing interpretable results. We further improve the performance of PseudoNAM by proposing PseudoNRSF, a random survival forest approach that takes as input the learned outputs of individual neural networks from PseudoNAM and predicts the survival probabilities. We show that PseudoNRSF achieves state-of-the-art results in terms of c-index and Brier scores.
Cox-based statistical and deep learning survival models
forests (Ishwaran et al. 2008) and multi-task logistic
regression (MTLR)
To address the censoring and interpretability challenges of existing survival analysis approaches, in this work, we propose two interpretable pseudo value based deep learning models, PseudoNAM and PseudoNRSF. Before describing our models in detail, we will briefly introduce pseudo values.
What are Pseudo values? Pseudo values for the survival
probability are derived from the non-parametric
populationbased Kaplan-Meier (KM) estimator, an approximately
unbiased estimator of the survival probability under
independent censoring
S^i(t ) = nS^(t ) (n 1)S^ i(t ) where, S^(t ) is the Kaplan-Meier estimate of the survival probability at time t based on a sample with n subjects and S^ i(t ) is the Kaplan-Meier estimate of the survival probability at time t
based on a leave-one-out sample with (n
1) subjects, obtained by omitting the ith subject. Pseudo values are calculated for both uncensored subjects and censored subjects (incompletely observed) at a specified time point.
PseudoNAM: Inspired by the success of pseudo value
based deep models, DNNSurv
+ f1(X1) + f2(X2) + + fp(Xp) (1) Here, Xi = (Xi1; Xi2; :::; Xip) is a p-dimensional covariate vector for ith individual; i = 1; 2; ::; n. g(:) is the link function (e.g., logit link function), is the bias and each fi(:) is parametrized by a neural network. y(tjX) is the pseudo values for survival probability at time t in the presence of covariates. Each of the networks learn the complex shape function of a specific covariate, and all the networks are trained jointly. A n
p matrix of p baseline covariates with n individuals are used as input in the input layer. Output layer returns the survival probabilities at M evaluation time points. PseudoNAM model provides interpretability because it jointly trains a set of neural networks corresponding to each individual covariate and returns the covariates’ contribution scores to the output (i.e., the output of the neural networks) for all each covariate and for the M evaluation time points. Then we sum up the contribution scores of all covariates followed by applying a sigmoid activation function to get the final output, i.e., the survival probabilities at the M time points. The non-overlapping neural networks for individual covariates allow to identify the individual covariate effect on the survival probabilities.
X1 X2 Xp . . . . . . . .
.
( ) ( ) . .
. ( )
{y11, y12, … , y1M}
{y21, y22, … , y2M}
{yp1, yp2, … , ypM} β
Add Sigmoid β Bias
PseudoNAM.
X fX1; X2; ::; Xpg is a p dimensional vector of covariates. fi(Xi) is the neural network corresponding to covariate Xi and
is the bias. Sigmoid ( ) is worked as inverse logit link function. y is the output, i.e., survival probability at M time = points. FC Layer means Fully Connected Layer.
PseudoNRSF:
PseudoNAM provides interpretable predictions; its performance is limited by the NAM
model architecture. To improve the performance of PseudoNAM model and to obtain global and local interpretations like Random survival forests (RSF) (Ishwaran et al. 2008), we propose PseudoNRSF - a two-stage deep learning model. In the first stage, PseudoNAM model is used to learn the individual feature contribution scores for predicting pseudo values. In the second stage, these learned feature contribution scores are input to an RSF with the goal of directly predicting survival probabilities. Thus, PseudoNRSF returns the subject-specific survival proban o it u b itr n o C e r u t a e F n a e M
Feature Importance Plot for METABRIC Dataset MKI67
EGFR
PGR
ERBB2 bilities at different time points as output, and the predictions are interpretable due to the use of RSF. PseudoNRSF has the interpretation property of the RSF. We can easily get the effect of each covariate (importance score) on the overall survival probability. However, using the PseudoNAM, we can see the change of covariate effect on survival probabilities at different time points.
We conducted experiments on three real-world datasets to answer the following questions: a) how well our proposed models perform compared to the state-of-the-art survival models? b) how well can our PseudoNAM explain their predictions?
Datasets: Table 1 shows statistics of the following datasets.
METABRIC: This data (Katzman et al. 2018)1 contains patients’ gene expressions and clinical variables for breast cancer survival prediction.
SUPPORT: This dataset (Knaus et al. 1995) is from the Vanderbilt University study to estimate survival of 9,105 seriously ill hospitalized patients.
WHAS: This dataset was collected to examine the effects of a patient’s factors on acute myocardial infraction (MI) survival (Hosmer and Lemeshow 2002).
Implementation Details: The (ground-truth) pseudo values for survival probabilities are obtained using the jackknife function of R package prodlim at each evaluation time point (separately for training and validation sets). We performed stratified 5-fold cross-validation so that the ratio of censored and uncensored subjects remained the same in each fold. We jointly train our PseudoNAM’s feature networks based on an early stopping criterion and choose the best model based on the model’s performance on validation data. Each feature network consists of 3 hidden layers with a number of units [128, 64, 32]. We used relu activation function in the hidden layer of each covariate’s neural network and tanh activation function in the output layer of the neural networks. In the final output layer, we sum up the output of individual feature neural networks and use the sigmoid activation function to get the survival probability at 10th; 20th; 30th; 40th; 50th; 60th percentile of the maximum survival time of the training data. We did not perform hyperparameter tuning. We set the learning rate 0.0001, output penalty coefficient 0.001, weight decay coefficient 0.000001, dropout rate 0.0, and feature dropout
EGFR
ER-positive
ERBB2
MKI67
PGR
rate 0. We used Adam optimizer with batch size 128 during
training and used Penalized Mean Squared Error loss
function to train our models. For performance metrics, we used
(a) time-dependent concordance index
Model Comparisons: We compared the following
survival analysis models:
• Statistical models: Cox Proportional Hazard Model
[CoxPH]
1944.0 205.5 0.1 1965.9 696.7 85.9 57.0 515.5 Max 355.2
Mean Median 100.0
Min 0.1
Censoring Time Max 337.0
Mean 159.6 344.0 2029.0 1059.9 371.0 1999.0 1298.9 1347.5
Median
158.0
918.0
sion [MTLR]
Table 2 shows the performance comparison of the survival models based on time dependent concordance index and Brier scores. From this table, we see that our PseudoNAM obtains similar or comparable performance to other survival analysis models, while PseudoNRSF outperforms all the survival models on the WHAS dataset, and obtains similar performance as the state-of-the-art models on the other two datasets. We notice that the independent neural networks for individual covariates limits PseudoNAM to learn the shared effect on the survival probability at different times, thus resulting in comparable but not the best results.
Model Intepretations The main advantage of our PseudoNAM models is that they can provide interpretations. Here, we discuss the two ways of interpreting the PseudoNAM model predictions: overall feature importance scores and feature-level interpretations.
PseudoNAM first learns the individual feature contributions for pre-specified time points (here, we choose 10th; 20th; :::; 60th percentile of the event horizon). Then we sum up these feature contributions followed by the sigmoid transformation to get the final survival probabilities at the pre-specified time points. Figure 2 shows the overall feature importance scores measured as mean individual feature contributions on the survival probability at the pre-specified time points for the METABRIC dataset. We see that the features can have a positive or negative impact (overall effect) on survival probability predictions at different time points for breast cancer patients. For example, the covariates such as MKI67, radiotherapy, and chemotherapy have positive feature contributions at the initial time points (10th percentile), which means that they influence better survival outcomes (higher survival probabilities). However, at later time points (such as 60th percentile), these features have negative feature contributions - meaning they result in mortality. This is expected because the survival probability remains higher at initial time points, and it decreases over time. Therefore, the treatment like chemotherapy fails to reduce the risk of death at later time points, and the older people (age at diagnosis) are at greater mortality risk.
Figure 3 shows the permutation feature importance, which is measured by observing how random re-shuffling of each covariate influences model performance. We use eli52 library to compute the feature importance for PseudoNRSF model. We observe that age at diagnosis has the highest importance on the survival probability predictions and ER-positive has the lowest feature importance.
Figure 4 shows the individual feature contribution
(i.e., the outputs of the individual neural networks of PseudoNAM) on survival probability at different time points (i.e., time-varying covariate effect on survival predictions) for the METABRIC dataset. Here x-axis shows the feature values, and the y-axis shows their contributions. In other words, this plot provides feature-level interpretations. For example, the survival probability for the feature age at diagnosis at all the time points starts decreasing after 65 years; and we see that the feature chemotherapy is biased to the patients who did not receive chemotherapy since the density is much higher for this group (darker brown bar). The plot also shows that the model predicted a decrease in survival probability for a few patients who received chemotherapy, especially at later time points. Why PseudoNAM is suited for healthcare domain? As shown in Table 2, our PseudoNAM models obtain good predictive and discriminative performance on all the survival datasets. Moreover, using our models, one can visualize each covariates’ contribution to the survival probability. Therefore, PseudoNAM helps to identify the potential risk factors for an event, such as death due to breast cancer. The visualization of the feature-level interpretations can be a step towards transparency in the deep learning models, which can inform clinical decision-making and perhaps lead to trust in the model. Thus, PseudoNAM models with high predictiveability and inherent interpretability could be well-suited for survival analysis in the healthcare domain.
In this paper, we proposed interpretable pseudo value-based deep learning approaches PseudoNAM and PseudoNRSF to model the nonlinear time-varying covariate effect on survival predictions. Our proposed models use 1) pseudo values to handle censoring and 2) neural additive networks to capture the complex nonlinear relationships and to obtain interpretable predictions. Empirical results show that our proposed models achieve similar or better performance than the state-of-the-art survival methods. Our PseudoNAM model provides both overall feature importance scores and featurelevel interpretations of predicted survival probabilities at different time points. For future work, we study and compare the interpretability of our proposed models with other parametric survival approaches.
This work is partially supported by grant IIS–1948399 from the US National Science Foundation and grant 80NSSC21M0027 from the National Aeronautics and Space Administration.
Gerds, T. A.; and Schumacher, M. 2006. Consistent estimation of the expected Brier score in general survival models with right-censored event times. Biometrical Journal 48(6). Hosmer, D. W.; and Lemeshow, S. 2002. Applied survival analysis: regression modelling of time to event data. Wiley. Ishwaran, H.; Kogalur, U. B.; Blackstone, E. H.; Lauer, M. S.; et al. 2008. Random survival forests. The annals of applied statistics 2(3): 841–860.
Katzman, J. L.; Shaham, U.; Cloninger, A.; Bates, J.; Jiang, T.; and Kluger, Y. 2018. DeepSurv: personalized treatment recommender system using a Cox proportional hazards deep neural network. BMC medical research methodology 18(1). Klein, J. P.; Gerster, M.; Andersen, P. K.; Tarima, S.; and Perme, M. P. 2008. SAS and R functions to compute pseudovalues for censored data regression. Computer methods and programs in biomedicine .
Knaus, W. A.; Harrell, F. E.; Lynn, J.; Goldman, L.; Phillips, R. S.; Connors, A. F.; Dawson, N. V.; Fulkerson, W. J.; Califf, R. M.; Desbiens, N.; et al. 1995. The SUPPORT prognostic model: Objective estimates of survival for seriously ill hospitalized adults. Annals of internal medicine .