Hepatocellular Carcinoma is a serious disease that can be caused of Hepatitis B virus-infected and lead to the death. Support Vector Machine (SVM) is a robust classifier method to predict the disease. However, unbalanced class data distribution is often effect to the performance of prediction and tend to identify the high volume class. Therefore, this research aims to ensemble methods by stacking the learning model of SVM with other classifier methods to increase the performance evaluation measurement. In the proposed method, two or three algorithms, i.e. Random Forest with k-Nearest Neighbor, and or Generalized Linear Model were ensemble to construct as a bottom-layer classifier model and were applied to the SVM model as a top-layer classifier. As a result, the performance measure of the proposed method was higher than the conventional SVM. The proposed method the accuracy rate of 89%, sensitivity of 87.2% and specificity of 82%. It slightly increased from the SVM as 2% for accuracy and sensitivity. However, the specitivity significantly increased as 82%. In nowadays, Hepatocellular Carcinoma (HCC) is making the government to give extra attention due to increasing the total amount of patients each year. The HCC was the third-ranking cause of death at 8.2% (781,531 cases) based on the Global Cancer Observatory database, the International Agency for Research on Cancer (IARC) (Global Cancer Observatory, 2019.). Many studies showed that the role of HBx in the pathogenesis of viral-induced HCC(Ali et al., 2014). By using the computational approach, the research on the patient's HCC that infected the HBx Hepatitis B virus was conducted by profiling the DNA sequence of the Hepatitis B Virus using the clustering method (Muflikhah et al., 2019). Support Vector Machine (SVM) is a robust classification method for HCC prediction. A lot of research was conducted using many types of datasets, such as image, clinical, microarray data of gene expression and DNA sequence (Ali et al., 2014a; Bai et al., 2018; Radha and Divya, 2016; Shen and Liu, 2017). However, the drawback of SVM is when applied to the huge data volume and unbalanced class. Ensemble methods are advanced techniques often used to solve complex machine learning problems. The method is a process where different and independent models as weak learners are combined to produce an outcome. The hypothesis is that combining multiple models can produce better results by decreasing generalization error. Stacking is one of the ensemble methods by combining many machine learning algorithms as a base layer of classification to predict the new data The research on the stacking machine learning model was conducted to improve accuracy in parallel computers (Gunes et al., 2017). Many other studies also conducted the ensemble construction to make a robust classifier than a single classifier to improve the accuracy and ROC (Abawajy et al., 2012; Buzhou Tang et al., 2010). Therefore, this research aims to detect the Hepatocellular Carcinoma using the nucleotide composition of HBx HepB virus using stacking learning model of machine learning to SVM. This paper is organized as follows. First, the background of this study and development of the hepatocellular carcinoma and detection of the disease in biological and computational approaches. Second, the research method including the proposed method using an ensemble method by stacking the learning model of SVM with other machine learning methods. The third section presents the results and discussion. The last section provides a conclusion and recommendations for further studies.
In general, the research was conducted with several steps as shown in Figure 1. The nucleotide composition was a result of the transformation from DNA sequence of HBx Hepatitis B virus database at URL: https://hbvdb.ibcp.fr/HBVdb/. First, the dataset was applied to three machine learning algorithm i.e. Random Forest, K-Nearest Neighbor (KNN), and Generalized Linear Model (GLM) including the SVM algorithm. Second, get the correlation between SVM to other machine learning methods and make sure the correlation value is not high. Then, applied the proposed method by stacking the the machine learning algorithms to SVM algorithm.
Random forest (RF) has been widely used as a robust machine learning method for classification and regression
or other purposes. The classifier is built based on an ensemble of decision unpruned trees randomly during training
Generalized Linear Models (GLM) extend the general linear model framework to address both of these issues: the range of Y is restricted (e.g. binary, count) and the variance of Y depends on the mean. A generalized linear model is made up of a linear predictor: = 0 + 1 1 + ⋯ + and two functions as this below Equation (Turner, n.d.)((Turner, n.d.): - A link function that describes how the mean, ( ) = , depends on the linear predictor ( ) = - A variance function that describes how the variance, var ( ) depends on the mean ( ) = ( ), where the dispersion parameter is a constant. 2.3
K-Nearest Neighbours (KNN) algorithm is a method that uses the data points are separated into several separate
classes to predict the classification of a new data point. The way in which the algorithm decides which of the points
from the training set are similar enough to be considered when choosing the class to predict for a new observation
is to pick the k closest data points to the new observation and to take the most common class among these
The Support Vector Machine (SVM) algorithm is initially a linear classification method that seeks the best function of hyperplane. The function divides two classes of input space, which are then developed into non-linear classifiers by incorporating kernel tricks in high-dimensional space. The data should be transformed into the vector space in a high dimension. The kernel trick functions that can be used in non-linear SVM classifications are Polynomial, Gaussian (RBF) and Sigmoid. Each label is denoted y¬i ϵ {-1, +1} for i = 1, 2, ..., n, where n is the number of data. The label is assigned +1 and -1 classes which can be completely separated from the hyperplane as defined in
The object data xi is assigned to +1 as in Equation (3).
.
+ = 0 . + ≤ −1 . + ≥ +1 1 ∥ ∥ (1) (2) (3) (4) The largest margin is calculated by maximizing the distance between the hyperplane and the nearest point In principle, a non-linear SVM concept changes the data x that is applied to the function Φ (x) in the high dimensional vector space. The objective function represents data in the new vector space. In the SVM, the learning process is finding support vectors by dot product of the new vector space data. The kernel function aims to determine a support vector for non-linear data in the SVM learning process. The kernel function can be stated as in Equation (5) . = Φ( ). Φ( ) (5) This research used the Polynomial kernel trick as shown in Equation (6).
⋅ = exp − − 2 2 2 (6)
The next step is to make predictions by implementing the Sequential Support Vector Machine method including
calculation of the Hessian matrix, iteration to reach the maximum in the least error rate or Max (| δα |) <ε. After
that, the bias and similarities between the testing data and training data are calculated. As a result, it will be
obtained the positive or negative classes
An ensemble is a set of classifiers that learn a target function, and their individual predictions are combined to
classify the new data. Stacking is an ensemble learning technique that combines multiple classification or regression
models via a meta-classifier or a meta-regressor. The base-level models are trained based on a complete training
set, then the meta-model is trained on the outputs of the base level model-like features. The base-level often consists
of different learning algorithms and therefore stacking ensembles are often heterogeneous. The algorithm in Figure
2 summarizes stacking
2: 3: 5: 6: 7: 9: 11: 12: 1: Input: training data
= { , } =
4: for t=1 to T do
Learn based on D end for
8: for i=1 to m do 10: end for 13: return H
learn H based on , }, where ′ = { ( ), … , ( )}
In stacking multiple layers of machine learning models are placed one over another where each of the models passes their predictions to the model in the layer above it and the top layer model takes decisions based on the outputs of the models in layers below it. The model predictions of various individual models are not highly correlated with the predictions of other models.
• •
The bottom-layer models (d1, d2, d3 ) that consist of Random Forest, K-NN, and Generalized Linear Model received the original input features (x) from the nucleotide composition dataset.
Top layer model, Support Vector Machine classifier, f() which takes the output of the bottom layer models (d1, d2, d3 ) as its input and predicts the final output.
Random Forest (d1)
Output (y) Hepatocellular carcinoma prediction Top layer model Support Vector Machine
f() K-Nearest Neighbor
(d2)
Nucleotide composition of HBx HepB Virus (x) Bottom layer model General Linear Model
(d3) Original input features
This research used the nucleotide composition of DNA sequence - HBx Hepatitis B virus in genotype-C as data
set. The regions of DNA sequence are representative of the nucleotide compositions such as Thymine (T), Cytocine
(C), Adenine (A), Guanine (G) at the first, second, and third positions of codon. The relative frequencies of the
four nucleotides can be computed for one specific sequence or for all sequences. For the coding regions of DNA,
additional columns are presented for the nucleotide compositions at the first, second, and third codon positions
2 3 4 .
n The composition of data set used is unbalanced class distribution, with 420 of HCC and 2862 of non-HCC as illustrated in Figure 4. The number of normal cases (non-HCC) is more than the number of carcinoma cases (HCC).
True positive (tp): The cases are predicted as carcinoma, and they are actually carcinoma. True negative (tn): The cases are predicted as not carcinoma (normal), and they are actually not carcinoma.
False-positive (fp): The cases are predicted as carcinoma, but they are actually no carcinoma. False-negative (fn): The cases are predicted as not carcinoma, but they are actually carcinoma.
Moreover, there are various measurements for performance evaluation, including accuracy, sensitivity,
specificity, and area under the curve (AUC), as presented in Table 3
Accuracy Confidence Level: 0.95
Kappa
Then, the correlation among the machine learning algorithms as sub models was evaluated to define as baselayer which is not high as shown in Table 5 and Figure 6. The highest correlation between svmPoly and Logistic Regression (GLM) at 0.475 is not high, so that recommended as sub-model layer of classifier. 0.87 0.86
Accuracy ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ● ●
In this research, there are four scenarios of stacking ensemble methods as bottom layer, details as follows: • Stack-1: Random Forest, Linear Regression and KNN • Stack-2: Random Forest, Linear Regression • STACK-3: Linear Regression, KNN • STACK-4: Random Forest, KNN
By using the stacking method as bottom layer for predictor and SVM with polynomial kernel for top layer, then the comparison of accuracy rates were shown in Figure 7. The accuracy is a measurement to know the ability of the classifier model to predict correctly. The proposed method, stack-1, stack-2, and stack-4 had the higher accuracy rate than the conventional SVM. The ensemble with Random Forest increased the accuracy of SVM algorithm. 6 7 8 . 0 1 7 5 3 0 . 0 S V M
Accuracy rate comparison of SVM against stacking ensemble methods
Sensitivity is a measurement to predict positive carcinoma correctly even though the related data is small size when compared to the normal data (negative class). The proposed method, stacking by ensemble method showed the sensitivity is higher than the conventional SVM method as shown in Figure 8. It means that it was not depend on the class distribution. 1 2 4 8 . 0 5 5 6 8 .
0 5 9 0 8 . 0 S V M
Furthermore, the specificity, the ability to predict in negative class, of SVM is highest as shown in Figure 9. The SVM methods can predict a negative class (normal, non-HCC) as maximum, due to high data volume in this class.
S V M
S T A C K - 1 S T A C K - 2 S T A C K - 3 S T A C K - 4
Finally, the AUC of the proposed method (Stack-1, Stack-2, and Stack-4) is higher than the conventional SVM using Polynomial kernel as shown in Fig. 10. It implies that the method can well classify to predict hepatocellular carcinoma disease based on the DNA sequences of HBx HepB.
Prediction of Hepatocellular Carcinoma disease was applied using the proposed method by stacking the learning model of Random Forest with k-Nearest Neighbor, and or Generalized Linear Model as a base-layer classifier model to SVM as a top-layer classifier. Because Random Forest has higher accuracy than other methods, then in stacking ensemble method, the prediction is based on the highest accuracy. In general, the performance evaluation result of the proposed method is higher than the Support Vector Machine in the single classifier model.
The proposed method needs high computation for the learning model due to the ensemble from many algorithms. In the future, it is possible to develop the selected algorithm with fast computation for ensemble learning model.
This research was financially supported by the Ministry of Research and Technology /National Agency for Research and Innovation (RISTEK/ DIKTI) in a program of Doctoral Dissertation Research Grant.