In recent years, credit card attrition has emerged as an issue of significant concern for the banking sector. It has a significant impact on profitability, given that the cost of acquiring new customers is higher than that of retaining existing customers. In this work, a selection of supervised Machine Learning models to identify which customers want to cancel their credit cards is evaluated. The banking industry uses this technology to obtain more reliable predictions when identifying opportunities for purchase, investment, or fraud. These models can be adapted independently, by recognizing patterns and algorithms based on mathematical calculations. Four models (LightGBM, XGBoost, Random Forest and Logistic Regression) were evaluated to predict, using data about customers and products held pertaining to a bank in Colombia, the likelihood of customers canceling their credit cards. By analyzing the ROC curves using the AUC metric, it is concluded that, of the selected models, the model chosen for deployment would be LightGBM, since it was the one that performed best in the experiments conducted. Furthermore, the “Score Acierta” variable, a customer rating provided by the Colombian credit rating agency, was found to be the most discriminating in prediction models.
The materialization of the constant risk of losing customers in banking entities has been a
considerable cause in the decrease of products and income for banks [
Their tactic has been to ofer low rates through portfolio purchases, expecting the balance to remain with the bank after the interest converts to the normal rate. However, many clients, hoping to maintain an attractive interest rate, transfer their balance from one card to another before the rate returns to normal. Likewise, there is an improvement in the condition of credit such as refinancing, lower monthly payments and consolidation.
As a result, the cost of acquiring new users is increasing. This has created a major shift in banks commercialization strategies, as many banks are focusing more on customer retention because it costs less to maintain an existing customer than to acquire a new one. In this way, a long-term customer tends to consume more and is less sensitive to competition. One way to improve retention is to take appropriate action towards customers at risk of loss or leakage.
Situations like these provide the possibility of creating a model through information sources and data analysis that predict the act of reducing and ending the use of a product or service after it has been activated. Attrition models, also known as retention or churn models, predict the probability that a customer wants to cancel some or all the products with the entity through patterns or sequences of financial activity.
The retention of clients is quite important since if they do not retain their loyalty their
acquisition cost is not compensated, and the expected profitability will not be achieved. To
determine it an analysis is made during a specific period of time where Equation 1 can be used,
which shows the retention rate [
Retention rate = (1 − Customer in portfolio ) ⋅ 100% (1)
Because these cancellations have such a significant impact on profitability, many companies are making these models the focus of loyalty strategies. To obtain the expected results, it is of great importance to have a standardized data set where accurate and truthful information can be acquired. The study was conducted in a Colombian financial institution, which did not have this new technology to develop the model and mitigate the problem. This has been a point that has made it dificult to develop the project while organizing a complete and functional data set that can yield results and the reasons for leakage.
As mentioned above, the bank did not have the necessary data to mitigate the cancellation or surrender of financial products. The areas in charge carried out a comparison of flat files, and the result obtained was a spreadsheet with the data of the clients that were likely to abandon it. Afterwards, it was sent to the marketing area to implement loyalty campaigns. It was a purely manual process and without reliability in the information.
For this reason, the need arises to make an attrition model that helps to focus the data and understand the situation of the bank’s customer churn, either due to the total and partial cancellation of the products, or inactivity for a long period of time. Having such a model will allow ifnancial institutions to define retention and create campaigns to carry out commercial actions in a “personalized” way.
In this work, four machine learning models (LightGBM, XGBoost, Random Forest, and Logistic Regression) are evaluated to predict through customer data and their products the likelihood of them canceling their credit cards.
This paper has the following structure. Section 2 discusses existing work addressing the
problem of attrition in banks. Subsequently, a subset of the phases of the CRISP-DM
methodology [
Van present a study carried out in England in 2003 [
In Manrai a multidimensional analysis was carried out of the satisfaction customers have
with the financial services of their banks [
It is noted that attrition prediction became the main focus of banks in China. Xie proposes a
learning method called Improve Balanced Random Forests (IBRF), which is applied to a data set
from a Chinese bank [
Verbeke indicates that customer attrition models must take into account two fundamental
components: the ability to predict and the ability to understand [
According to Chitra , for banks prediction of attrition is a subject of utmost importance. To
solve this problem, the authors have proposed using classification and regression trees to obtain
a better rate of categorization through the patterns of customers who left [
In China, to increase their profits from continuing operations and improve core
competitiveness, banks must avoid losing customers while acquiring new ones. In He the prediction
of customer attrition for commercial banks is analyzed using Support Vector Machines (SVMs)
and uses a random sampling method to improve the results of the model, taking into account
the imbalance characteristics of the customer data sets. The results show that this method can
efectively improve the prediction accuracy of the selected model [
In the United Kingdom in 2014 [
Zhao [
For the development of this project, data analysis was carried out using four recent algorithmic models, based on the implementation of the latest developments of each one, in order to predict a variable that identifies which customers want to cancel their credit cards. Furthermore, two techniques were implemented for the selection of definitive variables used to run the models.
The project was carried out in a Colombian bank with more than 50 years of experience in the market. This entity has more than 2.5 million clients nationwide and a wide portfolio of banking products, being one of the most important when it comes to banking Colombians. According to the economic activity of the country, the bank belongs to the tertiary sector: banking services, registering a 27% growth in loan portfolios, and increasing deposits by 29%. All these increases place it among the entities with the best growth in the bank-financial system, according to the Superintendence of Finance.
The analysis was carried out in the Customer Relationship Management (CRM) department and Analytics department, an area where all the information about the customer and their products is stored. With this data, models are developed that allow the attraction and retention of customers. The bank has its own cards with traditional franchises (generic cards and non-generic cards). The non-generic ones are associated with more than 30 brands across the country. Table 1 shows the distribution of the cards in the bank.
The type of cards chosen to carry out the project were generic. Although these make up a
lower percentage of the total distribution, more complete data about them is available, such as:
• Number of products in the bank with their respective balances and credit limits.
• Data such as the scores of the risk models to obtain the Recency, Frequency, Monetary
(RFM) model [
As non-generic cards are from private companies, their policies prevent sharing relevant information with the bank. For this reason, this type of card is not viable for the study to be carried out. The models made from the analytics area have the client as their main focus. Therefore, the data for this study will not be taken from each credit card but from the customers to which they belong.
The data obtained in Table 2 refers to the number of active cards that each customer has with the bank. In this way, we obtain the clients who are the focus of this study. An analysis was necessary to verify the cancellation percentage of credit cards at the bank. Table 3 shows how the behavior of the attrition in the bank was the quarter prior to the creation of the data set for the study. According to the analysis, it was shown that the percentage was steady but not especially high. However, this does imply a concern for the credit card area since the recurring cancellation of this product is being made on a monthly basis by customers.
The study uses a data set that contains a sample of approximately 220,000 records (active customers with generic credit cards as of July 2019). From this information, the financial behavior of the clients in the previous year (August 2018 - July 2019) was obtained. In this data set there is detailed information that the bank obtains from diferent sources to analyze the use of each customer’s credit cards, such as • The balance, the invoicing, the credit card limit, product usage and its cut-of dates; provided by the Analytics and CRM area • The scores provided by the risk area models. • The financial information of each client at the national level provided by the DataCrédito1 credit rating agency. This data is complemented with information on loans, current accounts, long terms deposits, mortgage loans, revolving loans, and demographic data (age, social stratification, gender, and economic activity).
There were three main data sources: the analytical area has its information in an SQL Server 2017 database, the risk area that has its data stored in an Oracle 11g database, and finally an Excel 2017 file sent from DataCrédito every three months. As can be seen in the Figure 1 SAS was used to integrate the sources and subsequently a .csv file was generated with the final data set.
At the same time, filters were made which allow obtaining cleaner and more relevant information for the case study. Therefore, the data set only contains credit cards that are active, that are not covered, nor provided by an SME or any type of company In addition to these filters, it is important to emphasize that it was not possible to diferentiate between voluntary attrition and involuntary attrition since the bank did not have the relevant information to obtain this diferentiation.
This results in the integrated data set, which is then analysed in order to determine the importance of the variables in terms of customer attrition, which will be used in the prediction models. For this purpose, variables are categorised thus: • Bank variables comprise those that capture the financial habits of customers, with the aim of identifying changes in the transactional behavior of users. Such variables include: the amounts of products per customer, balances (average and monthly) of the liability products, the credit limit, the number and value of credit card transactions, and number of months as a customer. • Sociodemographic variables are the measurable data that correspond to the general characteristics related to the personal aspects of the clients, among these are: age, economic
activity, gender, occupation, and social stratification • The target variable is the variable to be predicted that indicates whether a client canceled their credit card or not.
The subsequent step is to build a correlation matrix, using all the continuous variables from the data set. This matrix, shown in Figure 2, provided an understanding of the most positively and negatively correlated variables to determine which data were best outlined to be part of the data set used in the model. According to Figure 2 it can be observed that there is a high positive correlation between: the data of six and twelve months, the balance and usage of the credit card and the billing with the number of transactions in twelve months. Likewise, there is a negative correlation noted between the balance registered in other banks without a mortgage with the balance of all products with a mortgage.
Moreover, the analysis performed was made on categorical and continuous variables such as
(a) Histogram showing distribution of client ages
(b) Length of relationship with client in months
(c) Gender vs Attrition Diagram
(d) Occupation vs Attrition Diagram
age, where it was observed in Figure 3a that most credit card customers are in a range between
45 and 50 years. In [
The Figure 3b analyzes the client’s loyalty with the entity in months. In [
Likewise, Figure 3c indicates the gender distribution of the clients (female and male) against the response variable. There is a similarity observed in the percentage of canceled credit cards of both genders. However, it can be concluded that both genders behave diferently when managing money but their participation in the banking market has the same proportion.
Figure 3d shows the economic activity of each client. In this variable, the following fields were analyzed: independent workers, employees, and pensioners. It was observed that there is a scale in the cancellation of credit cards, where self-employed workers are the ones who most register this process. This is due to diferent factors that influence their economic activity such as having one or more products with the same financial institution and not being able to fulfill their obligations. Likewise, due to the nature of their work they are a more sensitive target to campaigns or ofers of other banking entities.
In Figure 4 the analysis was performed on the banking variable Quanto, which represents the estimate of customer income in DataCrédito. It can be seen that there is a diference and that customers who cancel tend to have lower income.
After carrying out the exploration on some variables, the analysis made was conducted to know the number of null values to determine if any variables got discarded for having more than 20% of null records. However, as can be seen in Table 4, no variable exceeds this threshold; therefore none were discarded. The table shows the six variables with the highest percentage of null records.
As a final part of the preparation, the selection of the variables to be used with the models
is made to identify which ones are the best qualified. Two techniques were considered for the
selection of variables. The first technique is Kolmogorov-Smirnov statistics (KS) [
( )|| where the main idea is to find the vertical diference in the distributions, then both functions are subtracted from the accumulated distribution, the maximum diference is obtained to see how diferent both distributions can be and the absolute value is used so that it does not matter that ( ) has a negative sign.
The second technique is the Information Value (IV) [
( Non Event% ) (3) This technique was the one that allowed the analysis of all the variables, since with it the number of bins could be modified and thus all the records of these variables were analyzed. Three functions were created to handle the diferent types of data in the data base. In Figure 5 the results obtained with the score given to each of the variables are observed, where the nine best qualified were chosen. With this result, the final data set obtained was used in the models.
Machine learning techniques are data approaches based on prediction and the construction of analytical models, with the aim of identifying patterns that can reduce the risk of leakage and have reliable results.
The study was conducted entirely in the Anaconda 2020.02 tool using the Jupyter Notebook IDE version 6.03 and Python version 3.8.3. The libraries with the versions used for the execution of the models are listed in Table 5.
For the analysis of the final data set, the following algorithms were taken into account in order to obtain the predictive model that best indicates which customers are going to cancel their credit cards and thus carry out marketing campaigns focused on customer’s needs.
Random Forests (RF) is an ensemble algorithm that uses decision trees as base classifiers, each
contributing one vote for assigning the most frequent class to the input vector. The RF increases
the diversity of decision trees by growing them from diferent subsets of data [
• Each tree grows to its maximum extent to obtain better results in the model predictions.
This algorithm allows the analysis of the understanding of the model through an inputoutput process, in order to obtain a number of variables to be selected in the participation of each tree. To find the parameters, it is necessary to leave one of the two variables with the value determined by the algorithm and the other will increase, in order to obtain the possible iterations.
The Extreme Gradient Boosting (XGBoost) [
The XGBoost algorithm is obtained through an initial tree 0 to predict the target variable
, which adjusts to the error of the previous step. The results of 0 and ℎ1 are combined to
obtain the tree 1. This process is iterative until the error is minimized as much as possible in
the following way [
Fm( ) < −Fm − 1( ) + hm( ) (4)
Using decision trees with low bias and high variance, this algorithm allows obtaining a real value score, independent of its classification and regression. This has the purpose of potentiating the results through the sequential process of the data with a loss function, which allows minimizing the error of iteration after iteration to become categories and to be able to build the next classifier.
Logistic regression [
The objective is to determine whether a variable or a set of explanatory variables have a coeficient equal to zero, in order to determine the parameters that must be estimated from the data to obtain the global adjustment of the model.
The Light Gradient Boosting Machine (LightGBM) [
Two cross-validation methods were used, stratified k-Fold, which is an improved variant of k-fold, since when dividing the data, it keeps the classes balanced, which is very important. The classification is a response variable, and samples with only 1 o only 0, will not be obtained. This method was used for the XGBoost, Random Forest and Logistic Regression models, in which five folds were used.
For the LightGBM model, lightgbm.cv was used, which is an option for this algorithm, the idea is to obtain more information about the estimation of the generalization error by evaluating the performance in a KFold division with fixed parameters.
As the last part of the study, the LightGBM, XGBoost, Random Forest and Logistic Regression models were applied to the training and test data sets that were obtained by dividing the final data set with the selected variables.
The graphs shown in Figure 6a and Figure 6b show the importance of the variables for two of the four models executed. In other words, these are the most relevant predictors. In both the Random Forest and LightGBM models, it was observed that the “Score Acierta” variable is the one that has the most weight. This variable discriminates best and that contributes the most to the models and refers to the Customer rating in DataCrédito. (a) Random Forests (b) LightGBM
On the other hand, the Receiver Operating Characteristics (ROC) curves were obtained with the area under curve (AUC) metric first for the training base. In Figure 7a the results obtained with this data set are observed where the XGBoost model was the one with the best prediction, being slightly superior to the LightGBM model, but to a greater extent than the Random Forest and logistic regression models.
However, when performing the same procedure on the test data base, a change was observed Figure 7b. as the LightGBM model had an AUC greater than that of the Random Forest, XGBoost and Logistic Regression models.
The ROC curves and the AUC metric obtained give the technical foundation to compare the obtained models, since their AUC on the test data was higher; the LightGBM model was chosen.
Attrition prediction models are important to reduce the cancellation of credit card products and analyze which customers have a greater tendency to cancel. This work was carried out through a strategic segmentation focusing on the client to generate campaigns according to the profile of each one, to understand and anticipate their behavior. Likewise, productive resources were focused on high-value groups, as it is cheaper to retain a client than attract a new one.
To generate a strengthening of the relationship between the client and the banking institution, understanding and anticipating their needs; also make the customer less sensitive to the competition to identify and quantify the impact of a retention program to understand customer output.
Future work could usefully include: (1) The evaluation of the selected models with a more updated, standardized data set and a wider time window; (2) Evaluation of other algorithms such as neural networks, genetic algorithms and SVMs to compare their performance with the results obtained; (3) Evaluation of attrition models for other products handled by the bank.