Artificial Intelligence is currently a popular theme in health care technology predictions. Machine learning is an artificial intelligence (AI) implementation that automatically learns and builds processes from experience. The leading cause of death worldwide at present is cardiovascular disease. The rate of death could be minimized by detecting the risk early. A lot of machine learning models have been developed to predict heart disease. We also introduced a fusion model to produce a better performance than the existing models. In this study, the proposed method analyzes two decision-making fusion models using ifve and ten-fold cross-validation to estimate the existence and absence of heart disease. The Jupyter Notebook, Scikit-learn, Tensorflow, and Keras were used as implementation tools. Three machine learning algorithms have been used here: the deep neural network (DNN), logistic regression(LR), and decision tree(DT). The decision tree and the logistic regression were merged separately with the deep neural network to form two fusion models (DNN+DT) and (DNN+LR), model-1 and mode-2. Five performance measurements have been used to compare model performance: accuracy, recall accuracy, f1-score, and AUC score. A significant improvement was found in performance parameters after fusing the algorithms in this work. The fusion occurred at the decision level by adding the decision scores of two algorithms. The main target is to enhance the fused model's performance by combining the individual model's decision for better classification. For both model-1 and 2, the accuracy has increased. Model-2 has obtained 87.12% classification accuracy for 10-fold cross-validation, which is the highest accuracy.
Models for Heart Disease Prediction 1. Introduction cular diseases (CVDs). CVDs are a category of problems of the cardiac and blood vessels, including stroke, heart attack, rheumatic heart disease, coronary artery disease, and other illnesses. Due to strokes and cardiac attacks, there are four out of five deaths, and people below the age of 70 die prematurely because of it, which is one-third unbalanced diet, smoking, stress, alcohol, fast foods, and inactive lifestyles. A study surveyed in 2016 that over 17 million individuals’ deaths are because of cardiovascular disease by the world health organization, which accounts for over 30 percent of deaths worldwide. The nEvelop-O LGOBE (S. Islam)
The good news is that heart diseases can be prevented
by avoiding some critical factors, such as poor diet habits
and insuficient physical exercise. To control their
general state of health and avoid sudden cardiac failure,
been established with the passing of time. To make the
privileged and medium-income countries is more than
same survey found that the number of mortality in under- to classify disease. Traditional approaches to treat heart
medical diagnosis easy and cheap, we choose to build a Following that, a study by [
This paper is presented as follows. The concern for architecture. The architecture of the neural network
conthe issue of heart disease is discussed in the first segment. tains 13 features derived from the Cleveland dataset. The
The second segment contains the previous work. Materi- proposed system gives 95% accuracy, and diferent
accuals and methodology are presented in the third segment. racy has been found with the variation of the number of
In part four, the paper explains the experimental out- hidden layers. The improvement in our work compared
comes of our suggested architecture for identifying heart those that we have used the fusion model to identify the
disease and also a comparative analysis with previous disorder, which results in improved outcome.
work. After that, we have ended with a conclusion and In [
A lot of work has been performed on the classification of heart disease. The goal of all researchers was to improve accuracy. Various algorithms have been applied to the dataset to observe which model performs best. Now the researchers are trying to build a fusion model for efectively classifying disease eficiently. Diferent algorithms are now making using fuzzy logic or merging algorithms to create a new one. Some of the relevant works have been discussed here, along with their accuracy.
In [
In another study by [
In a comparative study [
The dataset for heart disease was taken from the UCI ML repository [12]. Cleveland dataset was used, and the target of this study is to classify the patients with and without heart disease eficiently. This depository was developed in 1987 and currently holds 507 datasets. It has 13 features and 303 instances with some missing values. The dataset we used has been described in detail in table 1[12].
The possibility is pretty good that data collected from
any archive may have incomplete values or could even
contain outliers. The probability of missing data
values increases for the medical dataset [
After the preprocessing and cleaning of the data, data were cross-validated. Here the five & ten-fold crossvalidation was used, where the dataset was divided into ifve & ten equal parts. Four portions were used as training, and the left portion was used as validation at fivefold.
This study’s main objective is to form a fusion model with improving accuracy to classify cardiovascular disease’s 3.4. Feature Extraction presence and absence. By merging the decision of diferFeature extraction is the process of reducing the number ent algorithms using summation, a new fusion model has of attributes in a dataset. Irrelevant features that tend been developed with high accuracy. The Jupyter Noteto increase the model’s accuracy are reduced. Here the book, Scikit-learn, Tensorflow, and Keras have been used top ten attributes have been displayed in the figure 1, as implementation tools. contributing the most to the target output. In figure 2, the proposed approach contains three main those two algorithms’ output probability was merged to get a decision level fusion model’s output. After normalization, the data were trained with the three algorithms.
After training, we took two classifiers to make a new fusion model. The fusion was done by a simple summation of the probability score from two algorithms. If is the number of algorithms that we used in decision level fusion, and is the decision score from the individual algorithms after training. In that case, the final decision score for the fusion model can be represented by for each fold using the following equation.
= ∑ / =1
Input : Float value [][]
, two int and . [][]
is the decision score of a single algorithm of each fold, is the fold number in cross-validation and is the number of algorithms used in fusion fusion model Output : , the final average decision score for (2) 3 4 5 6 7 8 1 St=0; 2 for ← 0 to do = 0 end for ← 0 to do = [][] +
= / ; s c o r e o f e a c h f o l d 9 end 10 = /
/ / f u s i o n m o d e l ’ s d e c i s i o n = + ; / / s u m m a t i o n o f f u s i o n m o d e l ’ s d e c i s i o n s c o r e o f a l l f o l d s
A generalized algorithm-1 is displayed here. The sum of the decision score ( ) needs to be divided by the number of algorithms ( ) in fusion. As we took two algorithms for the fusion of model-1 and model-2, so we needed to divide the sum by 2. Using the final score, the fusion model predicted the output for test data. As both algorithms’ impact is present in our new fusion score, it decreases the rate for miss-classification. This process continues five times in five-fold and ten times in ten-fold cross-validation. After that, we took the average value from all folds and noticed an improvement in the average score comparing to the individual algorithm’s scores. To illustrate the procedure clearly, all fold values have been displayed in tables 2 and 4.
The fusion model’s output is based on two algorithms (DNN+DT) & (DNN+LR). We took the DNN and DT decision scores, and by summing them together, we got the fusion model’s decision score. This summation was done at every stage of the folds in cross-validation. At last, was used with back-propagation. It has been suggested as the performance of two individual models and fusion a very useful tool in diferent medical sectors for decision models was compared. This process is the same for both making. Figure 3 displays the structure of our proposed models for the same dataset with a diferent order. This DNN. new, improved fused model was used to diagnose heart disease.
A decision tree is a tree-like structure consisting of branches, nodes, and leaf nodes. It is a branching graph that functions like a splitting rule for each particular attribute.
Each feature is treated as a branching node. These nodes build a rule, and, based on the rule, values are grouped into diferent classes. In the decision tree, the leaf decides some decisions at ending, and the topmost is the Building A DT is easy and simple, and the results are predicted more accurately.
DT is sensitive to overfitting. It happens when the model is very good at identifying trained data but gives poor performance for test data. It becomes over skilled for training data by having minimal impurity in the leaf node. That’s why pre-pruning is needed to minimize the number of leaf nodes that are not that important for model building. It gives better accuracy for predicroot, which partitions the tree based on attribute value. Figure 3: Proposed Architecture of Deep Neural Network In this proposed approach, an input layer was used for data entrance, an output layer for predicting output(presence/absence), and two hidden layers. One of the most challenging tasks is model optimization for machine learning implementation. Model optimization reduces the test error; however, deep learning tunes the parameters outside of the model but has a significant tion. Information gain is another important criterion, influence on results and classification. The mother of and attributes with the highest information gain split ifrst. For that method, features with the lowest entropy are selected for splitting [13]. In the proposed design gini all hyperparameters is the learning rate. The speed of learning of the model depends on the learning rate. The number of hidden units is vital as it regulates the model’s was used as criterion, maximum dept of the tree was 8, representational capacity. The number of first and second and minimal leaf size was 10. Pre-pruning was done to get the best parameters. The Gini index and entropy are split criteria for a decision tree, and they are arranged by applying (3) and (4). hidden layers in our model is 10 and 8, respectively. L2 regularization was used to block overfitting. Bias, which is a constant, was used to help the model in a way that can fit best for the given data. Batch size ten was used.
∶ () = 1 − () = 1 − ∶ () = 1 − () = −
∫ =1 ∫ =1
2 log2 (3) (4)
An ANN has more than one layer, then is named a deep neural network (DNN). Like humans, a deep learning (DL) system can teach itself and learn through several hidden layers. DL is a model that is based on a multilayer feed-forward perceptron and uses back-propagation to train with stochastic gradient descent. There are four layers with nodes and neurons for the designed network, which has a uni-direction. It has two hidden layers, and there is a single way connection between every node and the next node. For training, stochastic gradient descent
LR is a supervised machine learning technique with continuous/discrete predictors. LR is generally used for binary classification. It is a statistical model that represents the relation between the binary dependent variable’s logit transformation and independent variables (one or more than one) by determining the best fitted linear model. This model is a simple prediction approach compared to other non-parametric models of machine learning with baseline accuracy scores provided by the model [14].
Heart disease has been classified using three ML models. After training the three models, two decision level fusion models have been developed using the trained models’ decision score. Five and ten-fold cross-validation was used for output performance. Five quality measures have been measured for the classification of the model.
Two decision level fusion models were constructed by applying an artificial neural network, logistic regression, and decision tree. Three algorithms were applied individually on the same dataset, and then the decision scores of the DNN and DT algorithm were combined as well as the scores of DNN and LR.
The fusion models (DNN+DT) and (DNN+LR) gave a better result than individual achievement. Tables 2 and 3 have shown the performance parameters of model1, and table 4,5 has displayed the outcome of model-2, AUC 81.70 each fold’s parameters improved than the separate model, which helped increase the overall performance of the fused model. Though the performance of fold-5 slightly decreased, overall, we found improved accuracy in the merged model.
Figure 4 represents the ROC curve of model-1 of every fold. Only the average value of ten-fold cross-validation has been displayed in table 3. It gave 84.21% accuracy for fused model-1, which improved with respect to DNN and DT.
ACC Prec RCL F1-score sAcUorCe
DNN 77.05 77 76 77 76.35 Fold-1 LR 78.69 79 78 78 78.14 86.15%, respectively. Also, it gave a better performance DNN+LR 80.33 81 80 80 80.86 than model-1.
DNN 83.61 84 83 83 82.95 We can see from all the values of tables 2, 3, 4 , and Fold-2 LR 83.61 84 83 83 83.23 5 that all the parameters slightly improved after the alDNN+LR 85.25 86 85 85 85.67 gorithms’ fusion in some folds. That is why the hybrid
DNN 83.61 87 82 83 82.41 model’s average performance is higher than the algoFold-3 LR 83.61 85 83 83 82.68 rithms’ average individual performance. Finally, it was
DNN+LR 83.61 87 82 83 86.52 found that model-2 (DNN+LR) had the best result for Fold-4 DLNRN 9816..6677 8972 8971 8972 9816..4513 10-fold cross-validation from all fusion models. And also, DNN+LR 88.33 88 88 88 88.17 for 5-fold, model-2 performed better than model-1. And
DNN 85 86 84 85 84.38 after fusion, improvement happened to all models. And Fold-5 LR 90 90 90 90 89.73 using these decision level fusion models, decision making DNN+LR 90 90 90 90 90.27 will be more eficient. As the decision probability of both
DNN 84.19 85.2 83.2 84 83.50 algorithms was fused, so the chance of more accurate Average LR 84.51 85 84.2 84.2 84.06 prediction increases. If one algorithm has a low probabilDNN+LR 85.50 86.4 85 85.2 86.30 ity for specific testing data (having heart disease). In that case, there is a chance that other algorithms will have a
Figure 5 displays the ROC characteristics of every fold higher probability, which will help the model to predict of the model-2. The roc curve of DNN, LR, and (DNN+LR) correctly. After merging the output probabilities, the has been shown in every diagram. model can easily predict the person with heart disease.
Table 5 shows that the average performance of 10-fold So, those data that have a chance for miss-classification cross-validation is shown, and it gave a good accuracy of may get correct classification after fusion, and thus a 87%, whereas DNN and LR individually gave 86.10% and better model will be constructed.
(a) Five-fold Cross Validation (b) Ten-Fold Cross Validation
This section discusses the work that has already been done with the same dataset as ours. In table 6, the output result of those works has been displayed with their approach. In every work, the researchers used various algorithms to classify disease. We have only shown the highest accuracy and the name of the algorithm to obtain that accuracy. Table 6 shows that the accuracy we found for our fusion model is a little more than their models. The outcome of both of our models using the same configuration has also be shown here. Some researchers used both five and ten-fold to calculate accuracy. Both results have been shown in the table to compare correctly with our work.
As we used both five and ten-fold cross-validations,
our accuracy was 85% for model-1 and 85.5% for
model2 using five-fold cross-validation. For five-fold
crossvalidation, the highest accuracy from table 6 is 83.83%.
So both of our fusion models performed well. The highest
accuracy for ten-fold cross-validation was 84.15%, which
was also obtained by a fusion model of MLP and SVM.
We got 84.21% and 87.12% for model-1 and 2, respectively.
So also, in ten-fold, our fused model gave better accuracy
for decision making. In paper[
At present, machine learning is a must, particularly in the health sector. This research aims to predict patients sufering from heart disease with greater accuracy using machine learning algorithms. Two fusion models have been developed to classify the presence and absence of heart disease. A comparative comparison was made to see which model performs better. Here, three algorithms have been used, and then two new decision level fusion models were created by combining the three algorithms. Both fused models gave a better performance than separate algorithms, and between the two fused models, model-2, which is the combination of DNN and LR, performed better than model-1(DNN+DT). Model-2 has an average accuracy of about 86%, where model-1 provided 85% accuracy with five-fold cross-validation. And for ten-fold, model-1 gave an accuracy of 84%, and model-2 gave 87.12%. In terms of other parameters, both models’ parameters improved. Both models’ performance is almost the same for five-fold cross-validation, and in ten-fold, model-2 performed much better than model-1. And in both models, we obtained improved accuracy after fusion, which was our main goal. The classifier can be used for diferent datasets for other medical diagnoses in the future, and the fusion model can be built with more than two algorithms.
Reseacrchers
I Tougui et al. [
Y K Singh. et al [16] S U Ghumbre et al. [17]
Year 2020 2019 2017 2016 2012
Approach
LR
LR SVM +MLP MLP
Fold number
Accuracy