In banking industry, customer churn prediction plays important role in business success due to the fact that the cost of attracting new customers is much more than that of retaining ones. Several Machine Learning (ML) models are being employed to make predictions in customer churn and achieve excellent performance. However, the problem with these models is a lack of transparency and interpretability. The goal of this work is developing explanations for customer churn prediction model using Shapley Additive exPlanations (SHAP) method.
Customer churn also known as customer attrition, customer turnover, or
customer defection, is de ned as the propensity of customers to cease doing business
with a company in a given time period [
Churn management is the concept of identifying those customers who are
intending to move their custom to a competing service provider[
Recent studies in predicting customer churn has investigated a wide range of algorithms. They are often evaluated on their predictive performance or their ability to discriminate between churners and non-churners. The performance of these machine learning models is remarkable but it is not the only aspect that is utmost importance. For customer churn prediction, understanding of the model and its outputs is important as well. It helps to target incentives to customers who have a high risk of churning to induce them to stay.
eXplainable AI (XAI) provides a suite of ML techniques that produce more
explainable models [
This work focuses on explanations with a state-of-art method, namely Shapley Additive exPlanations (SHAP) to explain customer churn prediction by several ML models. 2
In nancial service eld, customer churn prediction has been extensively
researched using di erent machine learning algorithms, particularly in banking
industry. Decision trees and logistics regression are widely used because of their
good predictive performance and robust results[
XAI has been applied in several elds such as healthcare, medical, and
nance[
In this work, we explore explanations of customer churn prediction models with SHAP method. 3
In this work, explanations of customer churn prediction models are performed in three steps. As illustrated in Figure 1 the data is rst preprocessed and then classi ed by several machine learning algorithms. Finally, the ML models are explained by SHAP method. Firstly, the data is cleaned by dropping irrelevant columns, handling any special values in the credit card customer dataset, converting text variables to numeric values, removing outliers, converting categorical values. Finally, the dataset is split with 70:30 ratios for training set and testing set. 3.2
After preprocessing data, most commonly used classi cation techniques are implemented to build customer churn prediction models. The implemented ML algorithms include Logistics Regression (LR), Gradient Boosting(GB), Random Forest(RF), and Nave Bayes. We also compare these models in term of their performance. 3.3
The main goal of this work is to enhance customer churn prediction models
by applying model-agnostic techniques. Model-agnostic methods are techniques
of explainable arti cial intelligence that can be used on any machine learning
models and are applied after the model has been trained [
Explanations can be divided into two types: global explainability or local
explainability. The distinction between them is in terms of scope. While global
explainability focuses on how the model works from a global view point, local
explainability targets speci c predictions from the machine learning model [
Several methods as widely used to interpret ML models: feature imputation
importance for ranking features that contribute to a model's decision, Local
Interpretable Model-agnostic Explanations (LIME) which focuses on training
local surrogate models to explain individual predictions [
output of any machine learning model[
In this work, we use SHAP method with KernelSHAP library because it
provides both global and local explanations and can be applied for all ML models.
The credit card customer dataset used in this work was collected from Kaggle [
The dataset has highly imbalanced classes, with attrition accounting for 16.07% of all customers.
Figure 2 is the correlation matrix between features evaluated in this work. Correlations are assessed using Pearson correlation coe cient. In this work, I used Scikit-learn libraries to experiment with several machine learning algorithms, namely Logistics Regression (LR), Gradient Boosting (GB), Random Forest (RF) and Nave Bayes (NB). (a) LR
(b) GB (c) RF
(d) NB Global explanations Figure 3 illustrates the global explanation results for the implemented ML models on the chosen dataset. Explanations for four models produce similar top features, however the rankings of features vary across all models. For instance, Gradient Boosting, Random Forest and Nave Bayes models agree that Total Trans Ct is the feature that a ects the model's outputs the most, while that of Logistics Regression is Credit Limit. These results are also consistent with top features related to the target variable in the correlation matrix in Figure 2.
Local explanation Local explanations provide a local understanding on how and why a speci c prediction was made. With SHAP method, explanations for a speci c prediction can be illustrated in a force plot. The red arrows in a force plot represent the features that drive the prediction higher, while the blue arrows represent the features that drive the prediction lower. The size of the arrows is proportional to the magnitude of the pushing force.
Figure 4 is the force plots for four implemented ML models which give explanations to a same single prediction. From the plots we can see that outputs of all implemented ML models are 0.0. This means that the customer is predicted to be not churn. However, features that impact to the outputs are di erent from models. In the RF and NB model, Total Relationship Count is the feature that major impact on the increase of output value, while that of GB and LR model is Total Trans Amt and Avg Open To Buy respectively.
(a) Logistics Regression (b) Gradient boosting (c) Random forest (d) Nave Bayes In this work, we perform global and local explainability for four implemented ML models using model-agnostic approach with KernelSHAP library. Given explanations help us understand the factors that result in customer churn. After performing the experiment, we realize that increasing the size of background dataset leads to increase runtime linearly as illustrated in Figure 5. This is consistent with the fact that Shapley values are basically estimated based on random sampling.
Within the scope of the paper, we only performed explaining ML models
of customer churn prediction but did not evaluate the given explanations. This
limitation of our research is due to the fact that evaluation has proven to be a
challenge[
In order to the success of machine learning algorithms adopted in banking industry, model explainability is necessary to ensure accurate results. The relevant literature proposed several methods to predict customers who are probably to move their custom to other banks. However, we found that exploration of the explainability was limited. In this work, we provide insights for explanations of customer churn prediction by using SHAP - a model-agnostic method. We also found that the runtime of SHAP method with KernalSHAP library increases linearly when increasing the size of background dataset.