Currently, most organizations are trying to build their data-driven strategies investing heavily in developing their own intelligent decision-making systems. But there are many small online retailers in the economy who are looking to implement business intelligence systems but still lack the necessary knowledge and expertise to do it. The article provides an example of using data science techniques for classification online store customers by their purchasing activity. The analysis of different approaches allowed us to propose the solution of this problem in two stages. At first, we segmented our e-commerce customers by RFM metrics using the k-means method. The algorithms for automated selection of the number of clusters and the initial selection of group centers are applied. There were 6 groups of clients highlighted: first cluster - lost clients; cluster 2 is a new wholesale buyer; cluster 3 - customers that the company may soon lose; cluster 4 active retail buyer; cluster 5 new retail customers; cluster 6 is an active wholesale buyer. In the second stage, with help of machine learning algorithms the customers' classification system was built. The presence of the second stage is conditioned by the need to take into account the constant updating of the client base and accumulation of new information. Tenfold crossvalidation was performed to avoid retraining models. The analysis of calcu?ations by 5 classification methods allowed us to give the advantage of the "random forest" method. To perform the analysis and all calculations this study uses R programming language and RStudio system.
Retail is one of the fastest-growing sectors of the Ukrainian economy. Today it is an almost unique market in Ukraine that has a lot in common with perfect competition. There are thousands of players in the segment that realize millions of different products. Most of them are small and medium-sized businesses. Year by year it becomes more and more difficult to win the loyalty of new customers and retain the loyalty of regular customers. Therefore, the ability to offer an individual approach to each of the clients in the coming years will be the only condition for successful business activity. Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
In view of the rapid economy digitalization, e-commerce is becoming one of the most important areas in retail activity. Despite the fact that Ukraine is far behind the global pace of e-commerce market development, in recent years’ Ukrainian online sales growth is even faster than in Europe. Nowadays, e-commerce companies have to refer to clients' wants and need in the decision-making process to meet the requirements of today's economy. At the same time, everyday contact with thousands of customers makes it difficult to consider each of them. The solution to this problem is facilitated by the development of a clear segmentation of the client base, which can be done based on the mathematical modeling methods.
Thus, modeling consumer behavior is an actual problem, which solution will not only improve the efficiency of e-commerce but also contribute to the development of the whole economy and better fulfilling of consumers' needs. In particular, the important task of e-marketing is to classify online store consumers by the level of their purchasing activity. The peculiarities of this task are a large amount of available data and their constant updating and accumulation, which requires the use of Data Science techniques, including machine learning methods.
The goal of this work is to classify online store customers by the level of their purchasing activity based on Data Science techniques, including machine learning methods. 2
The majority of Ukrainian scientists' researches is based on the analysis of the client
base, which is supported only by a personal understanding of the process [
Foreign scientific literature has many studies that reveal the peculiarities of the
usage of intelligent systems in marketing [
Also, the question of the further efficiency of the built algorithms is almost not
solved. Most of the research in this area details the methodology for clusterization's
existing customer base, but they do not take into account that new clients are coming
every day and current clients tend to change their behavior over time. It is considered
appropriate to consider clustering as the first stage of data analysis, which only allows
us to understand which customer groups are active, while it is important to have a
system for automatically assigning a segment to customers. In scientific research,
there are two approaches to solving this problem - fuzzy logic methods [
The approach proposed in the paper is implemented in 2 stages: the first stage involves customer segmentation by cluster analysis methods; the second stage involves the development of a client classification algorithm that would allow continuously update current clients segment and assign a segment to new customers.
This research is aimed to reduce the human impact in strategic decisions making. Therefore, particular attention is paid to the accuracy and relevance of the proposed methods and algorithms. The number of clusters is selected based on 26 different criteria and indices. For classification task were applied 5 different models with tenfold cross-validations. After that, the most accurate and appropriate algorithm was chosen for implementation.
Note that all calculations are performed using the R-Studio software with R programming techniques. 3.1
The study was performed on the sample of data from one of the online stores [
The database contains the following information: Invoice - unique operation code; StockCode - unique product code; Description - the name of the product; Quantity the quantity of purchased/returned products; InvoiceDate - date of operation; Price the price of the goods; Customer ID - unique customer code; Country - a country of the operation.
The first task to be addressed in the research process is the selection of criteria for
evaluating the level of customer purchasing activity. We will take a classic approach
to measure purchasing activity - RMF-model (Recency - Frequency - Monetary) [
Recency for each individual customer is calculated as the difference between the actual date in the database and the date of the customer's last purchase. In our case, the metric is measured in days. The frequency of purchases (Frequency) for each individual customer is determined by the number of transactions performed by the client during his client life. Monetary for each individual customer is defined as the total return on all customer transactions during his or her client life. In our case, the metric is measured in dollars.
In the previous research phase, these customer activity metrics were calculated for each customer in the sample. After that, the characteristics of the statistical distributions of Recency, Frequency, Monetary were calculated, namely: average, minimum and maximum values of indicators, as well as 1, 2 and 3 quartiles. The values of these characteristics are shown in Fig. 1. As you can see, the average customer of our online store had the last purchase 202 days ago. On average the customers buy 6 goods during the client's life while spending $ 2 720.
Further, we will use these indicators as the main metrics. The process of clustering an online store's customer base relates to Unsupervised Learning algorithms where algorithms do not receive any clues as to the desired result, but rather generate new results based on the data. Unsupervised learning technologies are commonly used at the beginning of the study. The main result of the implementation of these algorithms is to find certain patterns in the available data and to characterize their structure.
The most efficient and simple algorithm for cluster analysis is k-means. This method is very common in economic research, but its practical application for clustering e-commerce customers has some difficulties.
Firstly, the final results are sensitive to the initial random selection of group
centers. To solve this problem, a procedure involving multiple executions of an algorithm
with different random assignment of initial centroids was applied. An iteration with a
minimum value of is selected as the final clustering option. Within Cluster
Sum of Squares ( ) measures the squared average distance of all the points
within a cluster to the cluster centroid [
The sum of Euclidean distances between points within cluster l is calculated by the
formula:
( )
∑
(
),
(2)
where n is the number of points in cluster l; is the cent of the weight of cluster l
[
The second problem is the need to prioritize a fixed number of clusters for partitioning, which is certainly not always chosen to be optimal. Therefore, one of the main tasks of cluster analysis is to select the optimal value of k.
There are several versions of the solution: quantity is determined by business needs. This approach is commonly used if there is exist a proven customer classification system in the enterprise segment. An example would be the distribution of customers by their purchasing activity level (Low, Below Average, Medium, High); quantity is selected using machine learning algorithms. This approach is used when the decision-maker has no understanding of the typology of their clients. Machine learning algorithms help you to select customer classes based on the level of similarity of their behavior; a mixture of the first and second approaches. The most common approach is when a decision is made based both on business understanding and the results of mathematical modeling.
The basic methods of machine learning that help to solve the problem of choosing the
number of clusters are the methods of "elbow" and "medium silhouette". The elbow
method explores the nature of the (1) variation spread with an increasing
number of groups k. Combining all n observations into one group, we have the largest
intra-cluster variance, which will decrease to 0 as k → n [
Another, popular method of assessing the quality of the model is the "Average silhouette width". The value of the silhouette shows how similar the object is to its cluster compared to other clusters.
Suppose that the data were clustered into k clusters. For the point (the point ), let: (3) ; | ( ) | | ∑ ( ), where ( ) is the average distance from | | is the number of objects in clusters.
We can interpret ( ) as a measure of how well smaller the value, the better the destination).
Then we determine the average dissimilarity of the point the average distance from to all points (where , we now define: to other objects in the cluster is assigned to its cluster (the to some cluster as ). For each data point ( ) | | ∑ ( ), where ( ) is the smallest average distance to all points of any other cluster, where is not a member.
A cluster with this smallest mean difference is considered a "neighboring cluster" to , since it is the next cluster best suited for the point . Now let's define the silhouette of one data point : (4) (5) ( )
(( () ) ( ( ) )), where ( ) is the average distance from to other objects in the cluster ; ( ) is the smallest average distance to all points of any other cluster.
The silhouette varies from -1 to +1, where a high value indicates that the object is
well-matched to its own cluster. If most objects are of high value, then the clustering
configuration is appropriate [
By the formula (1) for each number of clusters (from 1 to 10), the level of variance explained by clustering was determined (Fig. 2). In Fig. 2 we need to identify the breaking point where the drop starts to slow. Points 3, 6 and 8 look most similar to the hacking point, but decisions made on one approach alone are in most cases not accurate.
Therefore, the next step will be a silhouette check. Using formulas (3) - (5), we calculate the value of "silhouette" for each variant of the number of clusters (from 1 to 10). A graphical representation of the calculation results is shown in Fig. 3. By this method, we look for the highest value of this indicator. As we can see in Fig. 3, the optimal number of clusters is 2 (6 in second place).
NbClust analytical package was used to refine the results, which allows us to calculate 26 additional criteria.
such that index > criticalValue
Smallest number of clusters such that index < criticalValue number of clusters such that critical value of the index >= alpha the cluster level before that index value < 1.00
Index
In Fig. 4 presents the number of criteria that supported the corresponding number of clusters. As we can see, the number of clusters in size 3 showed itself best (eight criteria selected this number). The next best option is to have 2 and 6 clusters. We can immediately discard option "2" as it will not bring us any value in future calculations.
Therefore, the main options are 3 and 6 clusters. The next step will be the practical implementation of the k - means method and validation of the results on business logic. The clients were divided into 3 and 6 clusters. Tables 3 and 4 show the average values of RFM metrics for the case of clusters 3 and 6, respectively.
The analysis of Table 3 allows us to give the following interpretation of clusters: Cluster 1. This includes customers who, on average, make small purchases every 2 months. Cluster 2. This includes wholesale buyers who, on average, purchase a large number of goods once a month for a considerable amount. Cluster 3. Here, retail customers make an average purchase once a year.
As can be seen from Table 4, the resulting clusters characterize the following types of clients: Cluster 1. Lost clients - Has made less than 2 purchases, the last of which was over a year ago. Cluster 2. New wholesale buyer - high average check and activity, but it's been a while since the first purchase. Efforts must be made to increase customer loyalty to the business. Cluster 3. Customers whose we will lose soon. They showed typical activity, however, a long time has passed since the last purchase. We should pay attention to these customers and try to persuade them to do more frequent operations. Cluster 4. Active retail buyer - high activity, buys for a long period, average check.
The most valuable and loyal type of customer for the business. Cluster 5. New retail customers - high activity, but during a short period, average check. Efforts should be made to turn them into regular customers. Cluster 6. Active wholesale buyer - high average check and activity, buys over a long period. The most profitable type of customers.
Given the business logic, it was decided that the division into 6 customer groups is more acceptable and better characterizes the current situation of the functioning of the online store. The number of clients in each cluster is shown in Table 5. 3.3
The next step after customer base segmentation is to build classification models for
the distribution of e-commerce customers by these segments. The classification is the
task of dividing the set of observations or objects by the values of certain attributes
into a priori given groups called classes. Within each of these groups, objects are
considered to be similar to each other [
The most common machine learning methods for classification are Linear discriminant analysis (LDA); Support vector machine (SVM), Classification and regression trees (CART), k - nearest neighbors (KNN), Random forests (RF).
Discriminant Analysis is a kind of multidimensional data analysis designed to solve random pattern recognition problems. It is used to decide what factors divide (“discriminate") certain data sets (so-called "groups").
SVM (support vector machine) is a set of similar supervised learning algorithms
used for classification and regression analysis tasks. A feature of the reference vector
method is the constant reduction of the empirical classification error and the intention
to increase the distance, so this method is also known as the maximum distance
classification method [
The decision tree develops solutions with the help of a tree model. The algorithm splits the sample into two or more homogeneous sets (branches) based on the most significant differentiators of the input variables. To select a differentiator (predictor), the algorithm takes into account all the features and makes a binary partition. He then selects the lowest cost option (the highest precision) and repeats recursively until the successful partitioning of the data across all branches (or reaches the maximum depth).
The Classification and Regression Tree (CART) is one of the implementations of the decision tree. Periodic nodes of trees of classification and regression are root and internal nodes - branches. The end nodes are leaf nodes. Each periodic node represents one input variable (x) and a splitting point on that variable; leaf nodes represent the output variable (y). The model is used to predict the following algorithm: it is necessary to go through all the splits of a tree in order to reach the node "leaves" and deduce the value present in it.
Random Forest (RF) is an ensemble model that builds several trees and classifies objects on a "vote" basis. That is, the object belongs to the class that has the majority of votes from all the trees. The algorithm trains several decision trees on different subsamples of data and uses the average to improve model prediction accuracy.
The K-Nearest Neighborhood Classification (KNN) algorithm assumes that objects
are divided into different classes so that they can be classified based on their
similarity. The distance between the objects may be a measure of similarity. KNN does not
need a training phase, it is trained in the sense that it begins to classify data points at
once, based on the majority of votes of its neighbors. The object is assigned the class
that is most common among its k nearest neighbors. [
The data set was divided into training and test samples (75% and 25% respectively). As a result, the training sample includes data about 3388 clients, and the test sample - 1130 clients. "Accuracy" was used to evaluate the quality of the simulation, which is the ratio of correctly distributed customers to their total. The clients are classified according to the 5 methods presented above (LDA, CART, KNN, SVM, RF). Tenfold cross-validation was applied during the implementation of the customer classification algorithm. It is necessary to test whether the simulation results are dependent on a particular dataset. A ten-fold test involves splitting the sample into ten randomly selected sets (test and training samples) and testing the model built on them.
Table 5 shows the characteristics of the Accuracy distributions (minimum, maximum and average values, as well as 1, 2, and 3 quartiles) obtained from the training sample for each method.
As can be seen from Table 5, the RF model showed the smallest error in the training sample (mean 0.99). Using a built "random forest" model, we check it on a test sample. The results of the customers' distribution by classes are presented in Table 6.
On the test sample, this algorithm showed an accuracy of 99%, so RF was chosen to implement the classification process for the entire data sample (Table 7).
Each cluster characterizes a specific group of customers that are similar in purchasing activity. At the same time, clients have a significant difference between clusters. A graphical representation of the difference between the level of purchasing activity in different clusters is shown in Fig. 5-7.
Segment t s a c e r o F 1 2 3 4 5 6 The paper deals with the task of classifying online store customers by their purchasing activity based on Data Science techniques, including machine learning methods. The analysis of different approaches allowed us to propose the solution of this problem in two stages.
First, the customers of the online store were segmented according to the k-means method by RFM indicators, using algorithms for automated selection of the number of clusters and initial centroids.
There were 6 customer groups found: cluster 1 lost clients - Has made less than 2 purchases, the last of which was over a year ago; cluster 2 new wholesale buyer - high average check and activity, but it's been a while since the first purchase; cluster 3 customers whose we will lose soon; cluster 4 active retail buyer - high activity, buys for a long period, average check; cluster 5 new retail customers - high activity, but during a short period, average check; cluster 6 active wholesale buyer - high average check and activity, buys over a long period.
The second step of the classification procedure, which is already directly carried out the classification of customers, due to the need to take into account the constant updating of the client base and the accumulation of new information. The analysis of calculations by 5 classification methods allowed us to give advantages of the "random forest" method.