Loan lending has been an important business activity for both individuals and nancial institutions. Pro t and loss of nancial lenders to an extent depend on loan repayment. Though loan lending is bene cial for both lenders and borrowers, it does carry a great risk of the inability of the loan receiver to repay back the loan. This inability is termed as loan default. Loan default prediction is a crucial process that should be carried out by nancial lenders to help them nd out if a loan can default or not. Successful loan default prediction can help nancial institutions to decrease the number of bad loan issues and eventually increase pro t. The aim of this paper is to use data mining techniques to bring out insight from data then build a loan prediction model using machine learning algorithms on the Sparks Big Data platform. Six supervised machine learning classi cation algorithms are applied to predict loan default: Logistic Regression, Decision Tree, Random Forest, Gradient Boosted Tree (GBTs), Factorization Machines (FM) and Linear Support Vector Machine (LSVM). Accuracy, precision, recall, ROC curve and F measure are used to evaluate the models and the results compared. We achieve the highest accuracy of 99.62% using the Decision Tree and Random Forest Models.
With increasing competition in the nancial world and due to severe nancial
constraints, taking a loan has become certain. Individuals and organizations rely
on loans for reasons such as overcoming nancial limits to achieve their personal
goals or for the basic purpose of managing their a airs in times where there
are nancial constraints [
Murray de nes loan default as when a borrower does not make required payments or does not comply with the terms of a loan. Pro t or loss of the nancial Copyright 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) lender to a large extent depends on loan repayments, that is whether customers are paying back the loans or not (defaulting). Therefore, when loans default,
nancial institutions will lose money, and it might even lead to bankruptcy
and collapse of the institution. By predicting loan default, nancial institutions
(lenders) can reduce credit risk, prevent loan default and increase pro t by
evaluating the ability of the borrower to deliver on their obligation of loan repayment
i.e. loan default prediction [
But with an ever-increasing amount of data for loan default prediction, there is the need to use Big Data applications. In this paper, we solve this problem by building a high-performing machine learning classi er model using Apache Spark machine learning libraries to predict loan default.
This paper aims to demonstrate the application of Big Data and machine learning in the nance industry. First, exploratory data analysis using data mining techniques is carried out to bring out insights from the dataset. Secondly, we employ Apache Spark machine learning libraries to make accurate loan detail predictions. Six supervised machine learning classi cation algorithms are applied to predict loan default, and we achieve the highest accuracy of 99.62% using the Decision Tree and Random Forest Models.
The structure of the paper is as follows. Section 2 presents related work.
Section 3 describes the research dataset, some exploratory data analysis and data preparation. Modelling is illustrated in Section 4. Section 5 evaluates and compares the presented models. Finally, Section 6 concludes the paper. 2
In this section, we review di erent types of mechanisms that have been employed for loan default prediction on di erent platforms/architectures.
Wang et al. present a study that uses 4000 samples and 21 attributes to build
and evaluate a classi er predictive model. Four algorithms are used: classic SVM,
Backpropagation Neural Network, C4.5 and R SVM. The result shows that the
total predicting accuracy of R SVM is better than other methods [
Reddy and Kavitha [
Hamid and Ahmed [
Odegua proposes the use of the Extreme Gradient Boosting algorithm called
XGBoost for loan default prediction. The prediction is based on loan data from
a bank with datasets containing 4368 samples and 10 attributes from both the
loan application and the demographic of the applicants. Location and age of
customers are the two most important features that a ect loan default. The
XGBoost model had an accuracy of 79%, precision (97%), Recall (79%) and F1 score
(87%). Conclusively, the paper provides an e ective basis for loan credit approval
to identify risky customers from many loan applications using predictive
modelling [
Mohammad et al. present a study on loan prediction by building a logistic
regression with a sigmoid function model and analysing the problem of predicting
loan defaulters. Logistic Regression models are built, and the di erent measures
of performances are computed. The models are compared based on the
sensitivity and speci city performance measures. The best-case accuracy obtained is
81.1%. The researchers made the conclusion that the logistic regression method
e ciently detects the right customers to be targeted for granting loans [
Patel et al. use various data mining algorithms to predict the likely defaulters
from a dataset that contains information about home loan applications, thereby
helping the banks to make better decisions in the future. The dataset used has
640,000 instances and 14 attributes. Optimum results are obtained using
Logistic Regression, Random Forest, Gradient Boosting and CatBoost Classi er.
CatBoost classi er and Gradient Boost provide almost equal accuracy with the
Gradient Boosting process giving better results of 84.035%. The researchers
concluded that these models can be used to make better decisions on loan applicants
in predicting loan default and save nancial institutions from undergoing huge
losses [
Meer uses a dataset consisting of 5,960 records. Two models are built using
tuned Logistic Regression algorithms, one model using a tuned Random
Forest classi er algorithm and one model using a tuned Gradient Boosting Tree
algorithm [
The dataset has been obtained from the Kaggle website and created for
pedagogic purposes for a common loan default prediction task. The data is generated
in such a way that default prediction machine learning models are likely to be
biased against women and minorities [
Figure 1 displays the scatter plot of income against education using pandas matplotlib python function available on Spark. The gure shows that as the educational level increases, the income of the applicants increases. So, it implies that the people with a higher level of education have higher incomes. This depicts a positive correlation.
Figure 2 shows the histogram distribution of the amount of loan size taken. For the rst 1000 rows of the dataset around 5000 is the highest amount issued.
Figure 3 displays the default class for the two gender types. Non-defaults (false or blue bar) are the highest meaning for both gender types, more people were able to repay their loan as compared to those that defaulted. In this section, exploratory Data Analysis using data mining techniques is carried out on the data to bring out some insights. Figure 5 shows the correlation Heatmap of the dataset. A correlation heatmap displays a 2D correlation matrix between two discrete dimensions, using coloured cells to represent data on what is usually a coherent scale. The rows show the values of the rst dimension, while the column shows the second dimension. The colour of the cell is proportional to the number of measures that correspond to the value of the dimension. Based on the heatmap diagram in gure 5, one can conclude that the target variable (default) is most positively a ected by some features such as rent and negatively correlated with job stability. 3.3
Data preparation is the process of preparing the raw dataset to be suitable for the machine learning algorithms. The initial pre-processing task entails null value removal and attribute data types adjustment. The data preparation steps are listed below: 1. Feature selection: The attributes that in uence the prediction are selected based on the correlation Heatmap and other social factors that in uence loan default. The attributes that are selected and used for model building
With the ever-increasing amount of data that is in gigabytes and terabytes
generated by the nancial institution to evaluate loan default, there is the need to
use Big Data applications that will e ciently and accurately predict loan default
no matter the quantity of the data. Apache Spark contains machine learning
libraries and is the most suitable Big Data application to carry out this task.
Apache Spark is a uni ed computing engine and a set of libraries (framework)
for parallel Big Data processing. It supports widely used programming languages
(Python, R, etc.), libraries (SQL, streaming, machine learning, etc.) and can run
from laptops to server clusters. Sparks provides a uni ed platform for
developing Big Data applications [
As explained earlier, loan default is the inability of a borrower to pay back his loan. So, when a loan default is true, it means the borrower has defaulted and cannot pay back the loan or meet up with the terms of the loan. However, if the loan default is false, it implies the borrower can meet up with obligation and pay back the loan.
Since the problem we are trying to solve aims to successfully classify values between two categories, true and false, the problem falls within the binary classication problem. Six supervised machine learning classi cation algorithms that are available on spark MLlib namely, Logistic Regression (LR), Decision Trees (DT), Random Forests (RF), Gradient Boosting Trees (GBM), Factorization machine (FM) and Linear Support Vector Machines (LSVM) are applied with the training data used to train the models and the testing data used to evaluate the models. Table 1 presents a brief description of the algorithms. We have used the Spark MLlib API on the Databricks environment to implement the models. 5
This section shows the model performance evaluation where 30% of the whole dataset is used for model testing and evaluation. Model evaluation is an important component of model development in which it evaluates the performance of the developed predictive models. Therefore, we carry out a comparison of the performance of the proposed models.
For model evaluation, we consider ROC curves, accuracy, recall, precision and F score derived from confusion matrices. A ROC curve is a graphical method that shows the sensitivity and speci city of a classi er model. A confusion matrix is a fundamental two-dimensional matrix that contains information about the actual and predicted categories of the classi er. Accuracy, recall, precision and F-score are then obtained from the confusion matrix parameters: true positive (tp), true negative (tn), false positive (fp) and false negative (fn). 5.1
Table 2 shows the overall evaluation metrics for the six developed machine learning classi cation models for loan default prediction. From the table, the accuracy
Algorithm Description Logistic Regression (LR) LR is a supervised classi cation algorithm used to predict a categorical response by predicting the likelihood of outcomes. It is a generalized linear model that predicts the likelihood of outcomes. Logistic regression is one of the features available in spark.ml that can be used to predict a binary or multiclass outcome by using binomial or multinomial logistic regression respectively.
Decision Tree (DT) DT is one of the most common supervised learning techniques used to solve classi cation problems. It has a structure of a tree with node and leaf representing features and class labels respectively. It is easy to understand, less data cleaning is required, and non-linearity does not a ect the model's performance, but it may have over tting problems.
Random Forest (RF) RF is exible and easy to apply supervised ML algorithm that produces, even without hyper-parameter tuning, a top-notch result maximum of the time. It can be used for both classi cation and regression duties.
Gradient-Boosted Trees (GBTs) GBTs classi er is a supervised learning classi cation algorithm. It is a collection of trees that trains a set of decision trees with "weak" constraints and uses boosts to combine predictions.
Factorization Machines (FM) FM is a supervised machine learning algorithm that is used to solve classi cation problems. Interactions between features are estimated even in problems with huge sparsity. The spark.ml implementation supports factorization machines for binary classi cation and regression.
Linear Support Vector Machine (LSVM) LSVM classi er is a supervised machine learning algorithm use to solve classi cation problems. SVM builds a hyperplane or set of hyperplanes in high or in nitedimensional space that can be used for classi cation, regression, or other problems. Intuitively, a good separation is achieved by the hyperplane that has the greatest distance to the closest training data points of any class. values for DT, RF, and GBTs are almost similar and are the highest (99%) but when critically examining the top values, the RF classi er model has the highest accuracy of 99.619%. We cannot measure the performance of machine learning models only based on their accuracy. In this research, other evaluation metrics like precision, recall and F measure are also considered.
As shown in Figure 6, the DT and RF classi ers have the highest ROC
curve value of 99.56%, precision 99.8%, recall 99.2% and F-score of 99.5%. It
implies that these two models perform better in loan default classi cation than
the remaining models. The main reason for this observation is that both DT and
RF works well for categorical and numerical values, also missing values does not
a ect their performance. Following these two models, we have GBT, LR, LSVM
and the FM classi ers with F-scores of 98%, 95%, 94% and 85%, respectively.
In this research, the factorization machine model is outperformed by the other
models. It might be due to the poor performance of FM on dense data. However,
FMs perform best in data with high sparsity [
As summarised in Table 2, the Decision tree and Random forest models show better performance (highest ROC scores) with a value of 99.5% while the Factorization machine model gives the least performance (lowest ROC scores) with 88.16%. This result is con rmed by the ROC plots displayed in Figure 7.
Figure 7 shows ROC plots for the DT, RF and FM algorithms proposed
in this paper. The Area Under a ROC curve (ROC) is the expectation that a
model will give an elevated grade to a randomly selected positive class data point
than a randomly selected negative class data point. A ROC curve that traces a
diagonal line signi es a poor classi cation algorithm, and it will randomly guess
if a loan will default. In addition, a good performing model will have ROC closer
to 1 while the ROC will be closer to 0.5 for poor-performing algorithms [
In this paper, the application of data mining and Big Data techniques in building loan default predictors is studied. Six models were developed using the Spark MLlib API allowing us to obtain the best performing model for loan default prediction. Based on the model evaluation, The random forest model presented the highest accuracy (99.619%). Also, the Random forest and Decision tree models have the best performance in terms of ROC curve (99.56%), precision (99.8%), recall (99.2%) and f-score (99.5%). Therefore, it can be concluded that decision tree and random forest classi er models are the most e cient and accurate in predicting the binary categories of loan default.
Based on the results obtained, Spark machine learning library-based models have shown a promising result in the prediction of loan default in this research. It allows nancial institutions (lenders) to be informed of default in issued loans beforehand which will help them reduce nancial loss and the cost associated with loan recovery. This will increase pro ts.