- An accurate prediction of prognosis to the patient diagnosed with Acute Myeloid Leukemia (AML) is an enormously difficult task. Several solutions have been proposed for prognosis prediction however there is a scope to improve current solutions. In this paper we aim at developing a solution that estimates the survival time that is the Prognosis of patients diagnosed with AML. To that end, we used a machine learning model that is built on an algorithm called Survival Regression. The model consumes as input the Expression Levels of a small number of the genes of the patient.
Keywords - AML, Gene Expression Levels, Machine Learning, Prognosis, Survival Regression
Acute Myeloid Leukemia (AML) is a form of cancer that
is characterized by infiltration of the bone marrow, blood, and
other tissues by proliferative, clonal, abnormally
differentiated, and occasionally poorly differentiated cells of
the hematopoietic system. Although it was incurable 50 years
ago, AML is now cured in 35 to 40% of adult patients who are
60 years of age or younger and in 5 to 15% of patients who
are older than 60 years of age. The outcome in older patients
who are unable to receive intensive chemotherapy without
unacceptable side effects remains dismal, with a median
survival of only 5 to 10 months [
We studied a number of existing analytics models for prognosis. According to our investigation, these models are built on the top of different statistical methods and machine learning algorithms. The study shows that the accuracy and correctness of the results produced by these models vary greatly. Also, we strongly believe that the accuracy level achieved by the existing solutions is low and can be improved.
Our objective is to develop and train an efficient machine learning model that can estimate the survival time, or the prognosis, of a patient diagnosed with AML, with high accuracy, using solely the Gene Expression Levels of a small group genes. The small number of genes means that the testing time and costs will be significantly reduced, as the gene expression levels of such number of genes can be measured using low-multiplicity technologies. The challenge will be to identify from thousands of genes the ones that their expression level can be used effectively to give an accurate prognosis. And the algorithm that the model will depend on is Survival Regression.
The rest of this paper is organized as follows: the second part is a background on some of the concepts needed for the understanding of the contribution. The third part is a literary view on the subject of machine learning in healthcare. The fourth part is our contribution and the fifth is results. The last part will be a conclusion.
Gene Expression is the process by which the information from a gene is used in this synthesis of a functional gene product such as proteins. Gene expression is contextdependent and is regulated in several basic ways: by region, dynamic response to environmental elements, by gene activity, and in disease states. So it can be said that gene expression level is one way of mapping his biological functions, and how well his body is being regulated by his genes. That is why the amount of gene expression, or the Gene Expression Level of the genes of a certain individual can give insight into his biological and thus medical profile.
Survival analysis is a branch of statistics for analyzing the expected duration of time until one or more events happen, such as death in biological organisms and failure in mechanical system. This is done by studying the relationship between the Explanatory Variables that lead to that event in a certain group and the Outcome Variable. For example, in cancer survival analysis, the event to be observed is the death of the patient. The time of survival of the patient is “regressed” against the variables that are under study, and the prognosis of the patient, or the duration of time before his death, is the outcome variable.
Survival models can be viewed as consisting of two parts:
the underlying baseline function, often denoted λ0(t),
describing how the risk of event per time unit changes over
time at baseline levels of covariates; and the effect
parameters, describing how the hazard varies in response to
explanatory covariates [
One of the branches of survival analysis is the survival
analysis under the Proportional Hazards Conditions. The
proportional hazards condition states that covariates are
multiplicatively related to the hazard. Sir Cox, whom the Cox
Proportional Hazard Model is attributed to, observed that if
the proportional hazards assumption holds, then it is possible
to estimate the effect parameter(s) without any consideration
of the hazard function. This approach to survival data is called
application of the Cox proportional hazards model. Cox also
noted that biological interpretation of the proportional hazards
assumption can be complicated [
Several works related to prognosis analysis have been found in a large bodies of literature. We reported some of the notable works related to our research.
Sara Haddou Bouazza et al. [
Feiyu Xion et al. [10] constructed a machine learning framework called KITML (Kernelized Information-Theoretic Metric Learning), which depended on the KNN algorithm but with improved distance metrics to diagnose cancer by finding similar patient biological profiles while dealing efficiently with the high dimensionality of microarray gene data. The performance of the algorithm was compared to others including KNN with Euclidian Metric, SVM, Random Forest (RF), and DT. The Macro Average F1 score was used as an evaluation metric and it showed higher accuracy of the algorithm, with considerable less execution time.
Sara Tarak et al. [11] classified cancer types by studying the Gene Expression Levels in 3 datasets: leukemia, breast, and colon using the Matlab Bioinformatics and Statistics Toolboxes. They used the KNN algorithm, with K=3 and built five classifiers each employing a feature selection method. The feature selection methods were chosen to avoid redundancy and noise and are: Backward Elimination HilbertSchmidt Independence Criterion (BEHSIC), Extreme Value Distribution (EVD) gene selection, and Singular Value Decomposition Entropy (SVDEntropy) gene selection. Tuned their results using Error Estimation, and evaluated them by using Receiver Operating Characteristic (ROC) and Bayesian Credible Interval (BRI) methods.
Kenneth R. Foster et al. [12] published a paper in which they discussed the great challenges facing the construction of classifiers in the field of bioinformatics, arguing they are mostly prone to over-fitting, and generally lack accuracy, and face a great challenge in the vast number of parameters that can interfere in the diagnosis process that are not taken into account. They stated that building classifiers should not be views simply as an add-on statistical analysis, but a parcel of the experimentation process, and that the validation of the classifiers for diagnostic applications should be considered as part of a much larger process of establishing the clinical validity of the techniques. The article mainly focuses on methods to improve the accuracy of SVM classifiers in bioinformatics unavoidable.
The technologies studied in the above tackle the problem of prognosis much lesser than diagnosis, as there are many solutions proposed on cancer diagnosis using different approach like use of medical imaging and Gene Expression Levels, however we found little about the prognosis problem. To the best of our knowledge, there is no solution proposed to tackle the problem of prognosis through the use of Gene Expression Levels alone, which have the potential to overcome the difficulty of producing accurate models, by eliminating much of the diversity in patients and their data, by focusing on data from a single, highly reliable source; their genetic profiles.
Furthermore, the implementation of the solutions proposed in literature the is done using classical solutions like Matlab and other statistical analysis tools rather than using advanced tools and technologies. This has limitations from a technical standpoint. Firstly, modification of any algorithm extremely difficult, very often not possible; the modeler relies on as is library provided by the development framework. Hence, improvement of a model is a non-trivial task. Optimization of performance in terms of computation is entirely impossible. Sometimes, technologies of these sorts provide some parameters to optimize performance.
However, very often it is not an effective approach. Specifically, for large-scale datasets, the existing solutions cannot be used meaning that the analysis cannot be performed these existing solutions. To the best of our knowledge, the scale of the dataset is critical to produce a comprehensive result of analysis – which is not possible with existing solutions. IV. PROGMOD – A MODEL FOR PROGNOSIS OF AML PATIENTS
In this section, we present ProgMod for finding the prognosis of AML patients using solely their gene expression levels. The process of model construction was sequential and iterative (shown in Figure 1), in order to find the model with the highest score. The number of features of different models varied as the aim was to find an optimal number that is preferably below 48.
The ProgMod development lifecycle consists of four phases: Data Preparation, Feature Selection, Model Construction, Model Evaluation. These tasks briefly described in the following.
1) Data Peparation: The dataset we worked with was of 240 AML patients, and is the combination of 2 datasets. 1) GSE 12417 (78 Patients) 2) TCGA (162 Patients). The information available about the patinets were three types: 1) Social Data (Age and Gender) 2) Clinical Informartion Data (Blast Count, Overall survival time osTime, and overall survival status osStatus) 3) Genetic Profile Data (Gene Expression Levels). The first step was to clean the data and remove the redundant information and extract the ones needed i.e. osTime, osStatus, and Gene Expression Levels.
2) Feature Selection: This was done in two different phases:
The process of applying variance thresholding to remove features that have low variance regardless of the value of the outcome variable i.e. the overall survival time. This reduced the number of featurs from 20000 to 287.
reduced variables the ones that best correlate to the outcome variable. Three features selection methods were used and tested in order to find the best one, they are: 1) F test 2) Recursive Feature Elimination 3) Mutual Info Selection. Sklearn’s feature selection module can be used for feature selection/dimensionality reduction on sample sets, either to improve estimators’ accuracy scores or to boost their performance on very high-dimensional dataset All the features that had a very low variance in 80% of the patients were dropped. The result of this process was to 1 http://facweb.cs.depaul.edu/sjost/csc423/documents/f-test-reg.htm 2 https://medium.com/@aneesha/recursive-feature-elimination-with-scikitlearn-3a2cbdf23fb7 reduce the number of features from about 20,000 to 287, these are the ones that will be used in the next step.
The second step is outcome reliant feature selection i.e. selecting the features that best correlate to the outcome variable, the survival time. First, and since the two dataset have different methods of measurements of Gene Expression Levels, they could not be used merged together to perform feature selection. Instead, the larger dataset, TCGA, containing 178 patients, was used as a features selection set, and the second dataset was used as a testing set.
Three feature selection methods were tried, evaluated, and the
one that had the best results was eventually used in the final
model. The three feature selection methods are all modules in
sklearn’s feature selection module. They are the modules:
- f_regresion: the f_test for regression tests whether
any of the independent variables in a multiple linear
regression model is significant using the ratio of
variances obtained from the means squared value. 1
- rfe_regression: recursive feature elimination
recursively removes features, builds a model using
the remaining attributes and calculates model
accuracy. RFE is able to work out the combination
of attributes that contribute to the prediction on the
target variable.2
- Mutual_info_regression: Mutual information is a
measure between two (possibly multi-dimensional)
random variables XX and YY that quantifies the
amount of information obtained about one random
variable, through the other random variable. 3
The pseudo-code for the algorithm used to select the best
model was the following:
for num_features in range (10, 48):
features = select_features
(num_features, Feature_Selection_Set)
model_score = evaluate_model
(features, Test_Set_1, Test_Set_2)
3) Model Constrction: Cox Proportional Hazard
Regression Model is used for developing analytics model. It
is probably the most popular regression technique for
regression analysis of survival data [
4) Model Evaluation: As mentioned the Cox Model predicts a survival function of the probability of the event of death over time. While this result is ultimately desired, it is difficult to evaluate the accuracy of the model when the real value is a number and the predicted value is a function. So an algorithm was improvised.
Each model was evaluated by fitting the data with the features resulting from the features selection process in to the testing set and scoring the model. The score of each model was acquired using the module r2_squard_accuracy in sklearn’s module: metrics. R2 Squared Accuracy is an algorithm used to predict how well the model will perform on future unseen data. It takes two matrices, one with the real values and one with the predicted values.
The real values used were the actual survival months of the patients. The predicted values were the values in the survival function corresponding to constant probability value. The probability value is irrelevant in this case because the purpose is comparison between different models. The predicted survival value for each patient was the value corresponding to 0.8 probabilty in the survival function.
ProgMod was implemented using different Python-based, open-source libraries available to perform data acquisition and transformation (Pandas and Numpy), analysis (Scikit Learn), and visualization (PyPlot).
For the implementation of the Cox Proportional Hazard Model, we used the open-source library lifelines, downloaded from the lifelines website4, and performed some tweaking and a wrapper around its functions to suit the needs of my implementation. Lifelines is a library that implements survival analysis models, including the Cox Proportional Hazard Model. https://lifelines.com4
Using the test set, the gene expression levels of the 47 genes selected in the previous step were fitted in the Cox Model alongside the overall survival time and the survival status. The model was the used to predict the survival function the patients.
For the purpose of scoring the model the value from the survival function of each patient corresponding to the fixed probabilty 0f 0.8 was taken, and was considered the predicted survival time for that patient. Then the array of real values and predicted values were used to score and visualize the result of the model.
Two metrics were used to score the model in each test set, both in sklearn’s module: metrics: - R2 Squared Accuracy: used to predict how well the model will perform on future unseen data. Its formula is: 1 − 5454∑∑:5:5;;44((66..7766<8<..))99 where Yi is the real value, Y’i is the predicted value, and Y’’i is the mean value.
average error per data point. Its formula is: =?> ∑C?D>( − <)B and predicted values.
where yj and y’j are the real Finally, the lifeline implementation of the Cox Model has its own score. When data is fitted into the model, it provides a summery containing a score called the concordance score, and it show how well the data fit the model.
For the test set GSE12417, we plotted the Real and predicted values to visualize which is shown in Figure 3.
0.56 72 1.39 56 1.98 07 1.77 41 1.71 29 1.65 38 The interpretation of the summary is the following: - Statistical Significance: The column marked “z” gives the Wald statistic value. It corresponds to the ratio of each regression coefficient to its standard error (z = coef/se (coef)). The Wald statistic evaluates, whether the beta (β) coefficient of a given variable is statistically significantly different from 0.
The Regression Coefficients: The second feature to note in the Cox model results is the sign of the regression coefficients (coef). A positive sign means that the hazard (risk of death) is higher, and thus the prognosis worse, for subjects with higher values of that variable.
Hazard Ratios: The exponentiated coefficients (exp (coef) = exp (-0.53) = 0.59), also known as hazard ratios, give the exact size of covariates.
Confidence Intervals of the Hazard Ratios. The summary output also gives upper and lower 95% confidence intervals for the hazard ratio (exp (coef)).
For Example, the genes 5604, 19432, 5521, 7596, and 445 have a p value < 0.001 and are the ones with the most statistical significance. These means that although we need all 47 genes to predict an accurate prognosis, these 5 genes are pretty good indicators of the prognosis of the patient.
In recent years, Bioinformatics is adopting mainstream technologies of computer science in performing various complex tasks. Machine learning – a branch of computer science- is one of the most popular fields of development of analytical models for performing analysis on critical, lifesaving subjects such as acute myeloid leukemia (AML). However, since there is an exhaustive number of algorithms and methods offered by ML, it is non-trivial task to find the best one without experimenting them. Furthermore, different models developed using different approaches produce different results with different accuracy levels.
We investigated different models proposed in literature for prognosis to the patients diagnosed with AML. According to our study, the results of these models vary which essentially imply that there is a possibility that the existing models could be improved in terms of accuracy.
In this paper, we presented a model to estimate the prognosis of AML patients using the Cox Proportional Hazard Model for survival analysis, using their Gene Expression Levels solely as the explanatory variables, thus overcoming the issue of high molecular heterogeneity of AML when it comes to prognosis prediction.
We have shown that it can estimate the survival time with r2_squared accuracy of 0.75 with the used dataset. And the model is valid to perform prediction on future patients. The number of genes that the model was constructed to work with is 47, a small number that with modern technologies can be tested with relatively small costs and time.
An extension of this work is lined up. That is, this approach can be extended to all sorts of cancer. However, the model has to be adjusted and generalized to select the significant genes, and perform the prognosis prediction of any cancer, based on the Gene Expression Levels of these selected genes.