The article presents results of the intelligent data clustering system for searching hidden regularities in financial transactions development. The main aspects and problems of increasing the volume of financial information within the client base segmentation for the formation of various development strategies and promoting goods in order to expand the target audience are given. The key opportunities and difficulties of using modern data mining methods and algorithms based on supervised and unsupervised learning are described and analyzed. Existing hybridization approaches implementation for data analysis algorithms, including those based on the use of data clustering ensembles, are considered. The concept of data analysis stages in the process of solving the segmentation problem is proposed, research metrics are formalized, clustering algorithms are selected and programmatically implemented via information system with the assignment clusters initial number and calculating it independently. Collected and formed balanced set of data on financial transactions for research, performed its statistical analysis, transformation and preparation for clustering. A software implementation of the system has been developed and its key functionality, component composition has been designated. The developed algorithms results studies based on the summary matrix of feature proximity analysis are presented, a unified space for cluster visualization is created based on the t-SNE approach, clustering quality assessing metrics are calculated. mining. Cluster data analysis, hidden patterns search, segmentation, financial transactions, data
Currently, there is a rapid increase in the number of non-cash financial transactions, both using
payment cards and through global payment systems with private currency units, electronic money,
and cryptocurrencies. This is especially noticeable in the global consumer electronic commerce
market through the use of online stores, social networks and other channels for the sale of products,
the provision of services or transfers between individuals or legal entities, in connection with which
the cash form of payment for goods and services is becoming less and less popular and up-to-date [
Making payment non-cash transactions process within the software or information systems used,
heterogeneous data sets are formed that represent the history of purchase transactions with reference
to time, location, type or costs category [
EMAIL:
2023 Copyright for this paper by its authors.
Such data can be valuable for businesses in terms of intelligent analysis (DM) of hidden patterns
based on the use of machine learning (ML) deep learning (DL) to build predictive models or customer
segmentation systems (by a number of characteristics, for example, by target audience or digital
footprint, portrait or behavioral image) in certain areas of human activity [
Identification of hidden patterns in the nature of payment transactions allows us to simplify and
improve the marketing process. In particular, the segmentation of the client base by separate groups is
advisable for the formation of personalized and most relevant to user expectations of promotional
offers, discount programs, special conditions and accumulative bonus systems, providing a higher
level of services provided as part of the competition for customers between players in the market of
goods and services [
The scientific and applied side of existing research in the designated area can be useful in the
context of development: directly of ML or DL methods and algorithms; development of analytical
systems and working environment capable of optimizing computing processes for data processing;
visualization and analysis results interpretation [
The complexity transactional data analysis lies in a number of aspects related both to the policy of
organizations that collect such information on its formation and provision, and to the technical
imperfection of the systems and technologies used. In particular:
1. in order to protect confidential information, some of the personal data of users describing the
completed transaction, which clearly identify the person, are hidden or inaccessible even at the
stage of their receipt, which complicates the formation of an exhaustive list of significant input
features [
All this determines the relevance of studying the possibilities of using hybrid intelligent ML technologies, methods and systems for analyzing financial transactions in order to search for hidden patterns in targeting and segmenting the customer base.
An analysis of modern publications focused number on DM questions on customer segmentation of
various organizations according to a input features number, including in the financial sector, made it
possible to establish the fact that approaches in this area are predominant based on the use of
algorithms and supervised or unsupervised models [
In the first case, the authors [
Most often, the authors in practice reduce the problem to considering the specifics of the
multiclass classification problem [
It should also be noted the different efficiency of the approaches used for different data sets,
which largely depends on the types of algorithms and metrics used to verify and evaluate ML models
quality [
In the case of using ML algorithms without a teacher, a number of identified problems are
leveled, but new aspects of model formation arise that need to be addressed. In particular, a number of
classical algorithms, for example, k-means, are fast in terms of computational performance, but most
often they do not allow achieving a sufficiently high accuracy level [
The mathematical formalization of the clustering problem can be represented as follows. Let X be a set of objects and Y be a set of numbers or labels of individual clusters. The given function for determining the distance between the available objects is a training sample of objects X m = {x1,..., xm} X .
It is required to split the existing sample into non-overlapping subsets, i.e. clusters, so that a separate cluster consists of various objects that are close in metric p , and objects belonging to different clusters differed significantly. For each object xi X m a separate cluster number is assigned yi .
The clustering algorithm is the function a : X → Y , which for each object x X puts in direct
correspondence the number of a separate cluster y Y . Set Y in some cases it is known in advance,
but it is necessary to determine the optimal number of clusters, in terms of a given criterion for the
quality of clustering results [
It should be noted that, from our point of view, the most appropriate approach is the analyzing data hybridization process on financial transactions based on a various combinations clustering algorithm [16]. Those, within the framework of the problem under consideration, it is advisable to use unsupervised learning algorithms, in particular methods of cluster data analysis, the advantages of which for the analysis of financial transactions are: • no need to markup financial data and simplify their pre-processing; • the ability to combine several algorithms within an ensemble in order to obtain better results and metrics; • the possibility of combining clustering algorithms, in which the values of the number of output clusters are initially set to select the desired target features in different ranges, as well as algorithms with automatic calculation of the number of clusters, which allows to ger results with additional verification; • eliminating the problem of introducing inaccuracies into the analysis results due to the presence of outliers; • distribution of computations among individual clusters based on the use of multi-threaded approaches when conducting experiments.
An additional advantage of this approach is the achievement of greater stability of the resulting solutions based on the adjacent ensembles algorithms formation by constructing a collective solution with many opinions (options for splitting data into clusters). The application of this approach makes it possible to reduce the dependence of the grouping results on the algorithm parameters choice, to develop more stable solutions in noisy data. The collective decision function combines each methods advantage used in the function’s construction [17].
When developing a collective decision, the results obtained by each individual model are grouped, because the more successful solution compensates for the less successful ones. At the same time, stable patterns, in accordance with which clusters are formed, are mutually enhanced, while unstable ones are weakened. The indicated approach makes it possible to carry out distributed computing in the case when obtaining a common database is impossible or not economically feasible. These properties of cluster analysis are especially relevant when working in areas with intensive data use, which is typical for the financial market [18].
Summarizing the results obtained, it is necessary to single out the main stages of cluster data analysis during the segmentation operation (in general, they are shown schematically in Fig. 1): selecting a sample for data clustering; determination of the required set of variables by which the objects in the data sample will be evaluated; calculation of values of the degree of similarity between the analyzed objects; application of a specific clustering approach to form groups of similar objects; results quality control. An additional relevant argument in favor of using this approach and developing our own system is ready-made data analysis systems lack that allow us quickly and accurately ensure clustering formation process with necessary algorithms selection. The currently existing clustering software systems in most cases are either private research works, which usually have fixed (not expandable) sets of algorithms, or commercial products, which are primarily focused on corporate clients use, which makes it impossible to have them for research and analysis [19].
Thus, it should be noted that the purpose of the work, which is to develop and study an intelligent system for identifying hidden patterns in transactional financial data, is relevant and in demand in practice.
The developed system is aimed at revealing hidden patterns in financial transaction data using the customers RFM segmentation by different types of groups in order to formalize the forming target audience process for further promotional offers development, demand management and customer base retention. 3.1.
Based on studies comparing a number of clustering algorithms popular in practice [17-23], it was decided to use 4 algorithms within the system being designed, 2 of which require an explicit specification of the number of clusters, and the remaining 2 calculate this value automatically.
The k-means algorithm, the idea of which is to minimize the distances between corresponding objects in separate clusters. The calculation process is stopped under the condition when minimization is impossible. Minimization by the k-means algorithm is carried out based on the following expression
M N J = d 2 (xi , ck ), (1) k =1i=1 where xi X - clustering object (single point in space); ck C - selected cluster center.
FOREL is a clustering algorithm based on the idea of combining objects into one cluster in the areas of their greatest concentration. The algorithm purpose is to divide the sample into an initially unknown clusters value so that the sum of distances from cluster objects to their centers is minimal. In fact, as algorithm implementation part it is necessary to select groups of objects as close as possible to each other, which, due to the similarity hypothesis, will form the final clusters.
k N F = p(x,W j ) (2)
j=1xK
The first summation is carried out over all sample clusters, the second summation is over all objects x , belonging to the current cluster K j , where W j - current cluster modified center, p( x, y) - distance between objects. DBSCAN is a density clustering algorithm, defined as a radius and the number of sample points in the generated range. In DBSCAN, each point (data object) in the sample is assigned one of the labels: the central point or the boundary (according to the specified radius R). Based on each core point or group of related core points, a separate cluster is formed (in the case when the distance between the core points is very close), after which each of the boundary points is divided into corresponding core points groups. Ward's hierarchical algorithm combines the results of previous algorithms to form a sample final clustering dendrogram. The general concept of forming an algorithms ensemble (fig.2) is based on the bagging approach, i.e. parallel models’ creation and their use on random data subsamples taken from a common pool. The final result definition for the objects in the identified clusters is determined by the voting of all ensemble models. The criterion for calculating the degree of similarity and clusters difference is the distance between the available points on the compiled scatter diagram, which corresponds to the distance between the points located on the graph. Within developed system framework, the method for estimating the Euclidean distance between points i and j on the plane is used ( Dij ). 3.2.
Dij =
(xi − x j )2 + ( yi − y j )2 To assess the accuracy and reliability of the proposed approach to clustering, the following metrics were chosen. Silhouette coefficient ( s ). Unlike other metrics, this measure does not imply initial knowledge of the true labels of the analyzed objects, and therefore allows us to evaluate cluster partitioning quality on an unlabeled sample, forming the final clustering result. Initially s defined separately for each object in the imported data selection. If a is the average distance from the given object to other particular cluster objects, b is the average distance from a given object to objects located in another (nearest) cluster, then the silhouette of this object will be the following value b − a s = max( a, b) .
In fact, the sample silhouette will be the objects silhouette average value included in a given data set. The convenience of this metric lies in the fact that it allows us to set the difference degree between the average distance to the cluster objects from the average distance to the objects of other clusters into which the sample is divided. The value of this metric is also in the range from -1 to 1, and the lower the score, the worse the clustering is performed (it is scattered). To determine the clusters optimal number Elbow score method is used, which is a heuristic used to determine the (3) (4) number of clusters in a data set. This method considers the change in the spread Wtotal with an increase in groups number k . By combining all n observations into one group, we can get the largest intracluster variance, which will decrease to 0 when k → n .
Kalinsky-Harabas index (for algorithms without explicitly specifying the number of clusters, i.e. FOREL and DBSCAN, since this metric allows you to additionally assess the degree of applicability of these algorithms to the selected data set), which can be used to determine the optimal value for clustering. The Kalinske Harabas index is defined as
Wtotal = min( k (1; n)) VRCK = SSB (N − K )
SSW (K − 1) , (5) (6) where K - clusters number, N - samples number, SSB - squares error sum between groups, а SSW - squares error intragroup sum. An additional factor is the algorithm profiling in the execution time estimation form. 3.3.
As an input data set, information was collected on customers financial transactions in online commerce, detailing their behavioral functions and reactions based on the open source number usage, including UNBANKS Real-Time EU Consumer Transaction Data. When the problem of unsupervised learning is solved, then the data are not initially labeled, so it is necessary to evaluate features similarity (proximity) to each other. For this purpose, a factor analysis was initially carried out to identify the most significant features to identify hidden patterns, the number of features was reduced from 255 to 15. Input features structure is divided into groups for the clustering process convenience.
1. Customers (general customer data): customer_id - client ID; product_X - product status. OPN opened but not disposed of. UTL - reclaimed. CLS - closed; gender_cd - gender of the client. M male. F - female; age - client's age in years; marital_status_cd – client’s family status (MAR married, CIV - civil marriage, DLW - does not live with spouse, UNM - single / not married, DIV divorced, WID - widower, widow); children_cnt - client children number; first_session_dttm - date and time of the first session in the application or personal account; job_position_cd - category of the position held (1 - service personnel; 2 - own official business; 3 - own unofficial business; 4 - head of the organization; 5 - senior manager; 6 - category not specified; 7 - housewife; 8 - individual entrepreneur; 9 - disabled person ; 10 - head of the initial level; 11 - temporarily unemployed; 12 old-age pensioner; 13 - disability pensioner; 14 pensioner; 15 - seniority pensioner; 16 - nonmanaging specialist employee; 17 - head of the unit; 18 - unknown ; 19 - works for individual entrepreneurs; 20 - non-executive employee-worker; 21 - working pensioner); job_title - client's position.
2. Transactions (data on completed transactions): customer_id - client ID; transaction_month month of transaction; transaction_day - transaction day; transaction_amt - transaction amount in dollars; merchant_id - ID of the store where the transaction was made; merchant_mcc - MCC category.
The total data sample size is about 50,000 customer records and more than 1 million transaction records. For pre-processing of the collected raw data, they are aggregated, recorded and saved in *.txt format, and also structured as two separate samples according to customer data and transactions for sequential transaction analysis convenience. After saving and formatting the raw data in the *.csv format, they can be visualized in a tabular form convenient for interpretation and analysis. In order to conduct a preliminary exploratory analysis, a statistical study of the collected data was carried out, fragments of the results for the first sample are shown in Fig. 3.
The prepared data is divided into equal fragments for the purpose of their distributed processing within ensemble using framework to simplify the procedure for their random distribution between clustering models.
Figure 3: Summary analysis of sample data
For object-oriented software application operation modeling, we use UML methodology and notations. The main options developed diagram for software application using is shown in Fig. 4.
The user can carry out: import of input data sets; factor analysis; preprocessing and cleaning of data from outliers; input features choice; building a diagram by features; filling in gaps; formation of summary statistics; data normalization; data clustering by one or more of the selected algorithms (Forel, k-means, Ward, DBScan) as assemble; building visualizations (dendrograms, error matrices and scatter plots); assessment of clustering metrics; formation of a table of segmentation results.
After importing the data, the input datasets are loaded, collections are formed to store the imported records, after which exploratory analysis is performed, including feature analysis, the formation of additional average features for data comparison, and dataset key features distribution visual diagrams construction.
Based on the formed data structure, missing values in the data set are eliminated, records are aggregated into a single set. After that, the normalization objects are set for each of the analyzed features. Each clustering algorithms parameters are configured based on the performed normalization, after that the distances between the features in the generated state space are estimated. Next step is clustering metrics calculation and the results are displayed in graphical visualizations form. The main application, when forming an clustering algorithms ensemble, uses a number of external dependencies (scipy for hierarchical clustering; sklearn, NumPy, Pandas for building data structures and clustering algorithms; matplotlib and seaborn for visualizing clustering results in a graphical form) and own FOREL and Transformers objects for implementation Forel algorithm logic and data normalization, respectively. The application deployment diagram is shown in Figure 5.
Local development is done using the Python programming language version 3.8 and the PyCharm development environment. By using the Google Chrome browser, when authorizing through a gmail account, one enters the Google Colab cloud web service for deployment, testing and research of the system in the form of an *.ipynb file. For convenience, data is loaded directly from the cloud storage directly into the system upon request. Within the framework of the developed system, it imposes a number of requirements on input data samples, which is why a number of preliminary checks are implemented when importing data, in particular: • verification of the absence of a direct correlation between features (correlation coefficient must be less than a user-specified threshold, 0.5 by default); • verification of dimensionless form of data; • distribution nature assessment for similarity to normal; • indicators stability; • sample homogeneity.
In order to carry out a created clustering ensemble work generalized assessment a number of studies were carried out on various transactions volumes. For the k-means algorithm, the values of the following hyperparameters are set: the number of clusters n_clusters in the range from 4 to 9, the number of setting different centroids n_init = 'auto', the computational iterations maximum number value max_iter in the range from 200 to 300, relative tolerance tol = 1e-5 , a variation of the algorithm chosen by lloyd and elkan.
The following hyperparameters are specified for the DBScan algorithm: the maximum distance between two samples eps in the range from 0.3 to 0.7; number of samples in the vicinity of the point min_samples = 4; proximity calculation metrics metric = euclidean; version of the comparison algorithm algorithm = brute; number of simultaneously running computing processes n_jobs = 4.
For the Forel algorithm, the values of the search radius parameter for local concentrations were set using sliding control.
On the basis of the conducted studies, input data individual categories proximity indicators relative to each other were formed and evaluated and are expressed in relative units.
The generalized matrix for average values analysis by features in clusters relative to the global average is shown in Fig.6. Сlusters numbers selected by the algorithms are plotted along the y-axis, the main features are indicated along the x-axis, and a color scale is additionally introduced to visually display the features similarity degree in clusters. Some fuzziness between 4 and 5 clusters should be noted. Due to the fact that the results of the clustering ensemble are not linearly separable data due to large dimensions presence, it is necessary to compose mixtures of objects that will belong to normal distributions with different parameters using the t-SNE algorithm, which will ensure multidimensional space decomposition to the appropriate level. This process is resource intensive and was performed in a distributed mode based on the use of 4 separate threads and took about 50% of the total duration of all computational experiments. Result visualization obtained based on developed clustering ensemble use is shown in Fig. 7. As we can see from the results obtained, clusters 2 and 1 are more focused and separated from the rest. The greatest blurring character corresponds to clusters 3 and 5, which may be due to the correlation between a number of secondary features that affect the generated transactions and are not separated into separate subgroups. Based on a summary assessment of all algorithms implemented within the clustering ensemble, a table was formed with the obtained metric values results assessments and overall performance in the computing operations profiling time form (Table 1). Due to the specifics of the analyzed data and taking into account a hidden dependencies number, clustering algorithms do not always clearly and unambiguously segment customers. However, the analysis of the operation of clustering ensemble algorithms made it possible to identify some commonality in a number of features, which is the basis for identifying a customer’s categories number. In particular, categories can be distinguished by 6 user clusters (presence/absence of children, over/under 45 years old, exceeding/not exceeding the average monthly salary for the position, male/female, married/never married, shopping on weekdays/weekends).
Based on the results obtained within the considered data set, it is reasonable to note that the identified 6 customer clusters, taking into account their purchases according to the MCC code, should be attributed to the following named categories: housewives, businessmen, models, transport employees (logistics), tourists and students (students) .
Within the framework of the developed system, it is easier and more convenient to configure and evaluate the k-means clustering algorithm (which is the fastest, but there is a greater spread in the values of metric estimates), since dbscan and FOREL are highly sensitive to changes in hyperparameters, and therefore their time costs are higher. The k-means and FOREL algorithms provide more balanced clusters. Analyzing the differences in average values in different clusters, we can see that the algorithms often separate customers by product (for example, product 6), but not so often by gender. Customers are better separated by transaction-related parameters, such as how expensive the goods are in stores, and how much money the customer spends relative to other customers in a similar role. By visualizing average values in clusters relative to global averages, it is possible to identify relationships between the several parameters manifestations at once, especially if these relationships are detected by several algorithms at once.
The relative maximum discrepancy between the algorithms in clients number terms in the cluster is quite insignificant and is about 3.5% for group 2, the smallest discrepancy values are noted for groups 1 and 5. Such values are an additional side sign input features choice correctness as a result of factor analysis and as a clustering high accuracy result.
The approach to data analysis proposed in this article and the developed system that provides software support for the described algorithms with the ability to customize them and build an clustering assemble, makes it possible to automate identifying hidden patterns process in data by combining unsupervised learning algorithms. The use of hierarchical and non-hierarchical data clustering based on the ML models’ ensembles usage allows multi-level customer segmentation in a multidimensional feature space.
The created system can be used for data segmentation tasks with different structure and application area due to its versatility in data preprocessing functionality terms and results visualization.
A further target proposed approach developing way is the integration within the system of other clustering data analysis algorithms, as well as methods for forming clustering ensembles based on boosting or stacking, followed by the development of a functional to compare their effectiveness with each other on the same data sets. Also, it is advisable to supplement the system with the ability to support other metrics for evaluating clustering quality and computing operations process perform profiling with results visualization by the consumption level hardware resources in online mode.
[16] R. Burton, S. Cuff, M. Morgan, A. Matt, M. Eberl, GeoWaVe: Geometric median clustering with weighted voting for ensemble clustering of cytometry data, Bioinformatics (Oxford, England), (2022). DOI:39. 10.1093/bioinformatics/btac751. [17] Z. Wang, H. Parvin, S. Qasem, B. Tuan, K. Pho, Cluster ensemble selection using balanced normalized mutual information, Journal of Intelligent & Fuzzy Systems, 12 (2020) 1–23. DOI: DOI:10.3233/JIFS-191531 [18] S. Bi, W. Liu, Clustering Analysis of Online Teaching Cases and Evaluation of Teaching Results, International Journal of Emerging Technologies in Learning (iJET), 18 (2023) 128-142.
DOI:10.3991/ijet.v18i03.38055. [19] H.Li, X. Ye, A. Imakura, T. Sakurai, LSEC: Large-scale spectral ensemble clustering. Intelligent
Data Analysis, 27 (2023) 59-77. DOI:10.3233/IDA-216240. [20] T. Uçar, A. Karahoca, Benchmarking data mining approaches for traveler segmentation, International Journal of Electrical and Computer Engineering (IJECE), 11 (2021) 409-416. DOI: 10.11591/ijece.v11i1. pp. 409-415. [21] S, Rogic, L. Kascelan, Segmentation Approach for Athleisure and Performance Sport Retailers Based on Data Mining Techniques, International Journal of E-Services and Mobile Applications, 13 (2021) 71-85. DOI:10.4018/IJESMA.2021070104. [22] A. Gunandi, H. Awang, E. Alhawad, L. Shabaan, Customer Value and Data Mining in Segmentation Analysis, International Journal of Information Technology and Computer Science Applications, 1 (2023) 20-34. DOI:10.58776/ijitcsa.v1i1.16. [23] C. González, R. Delgado, J. María, S. Santos, Segmentation of Potential Fraud Taxpayers and Characterization in Personal Income Tax Using Data Mining Techniques, Revista Hacienda Pública Española, 239 (2021) 127-157. DOI:10.7866/HPE-RPE.21.4.4.