To increase the effectiveness of detecting fraudulent bank transactions, the structure of the system is proposed to analyze data of user environment in order to identify potential fraudulent activities. The system for collecting and analyzing information about the user environment allows to accumulate data about the user environment, to mark precedents in manual and automatic modes and build a database of images for classifiers training. It is necessary to implement data collection, storage and access interface for the application of data mining tools. Operation of significant amount of accumulated data requires the use of special tools (frameworks and hardware platforms) for processing large data. In this paper the analysis of the existing software and hardware tools for distributed processing of indefinitely structured data of bank transactions (frameworks: Hadoop, Apache Spark) is presented. The structure and recommendations for the deployment of a hardware and software stand for testing algorithms for detecting financial fraud on the basis of data mining analysis as part of a distributed data processing system for bank transactions based on the selected framework are developed.
The penetration of information technologies into all spheres of human life creates the basis for the
formation of new conditions for the functioning of the market. In these conditions it became possible
to develop the digital economy. The key factors of this economy are electronic technologies and
services and digital representation of large volume multi-branch data [
An important aspect of the functioning of the digital economy is the provision of information and
economic security of business, personal data protection. As a result of the rapid development of
financial technologies in the whole world, there has been an increase in fraudulent activities in the
electronic environment. According to the Central Bank of Russia for the year 2014, the share of
fraudulent transactions in Internet banking was 63%, and in the last 2 years – increased 5.5-fold and
accounted for 93% of all crimes related to embezzlement of funds from cardholders' accounts [
The most promising solution for today is the use of technologies to define the user environment in
combination with the methods of machine learning. The use of machine learning is a necessary
metrics, since a large amount of information about the user environment is collected and the
application of rules to this data becomes impossible. Classic methods for detecting fraudulent actions
can not sufficiently accurately answer the question: was this action actually performed by the user?
There are many ways to obtain illegal access to a user's account: phishing, vishing, pharming, mobile
fraud, and other methods related to social engineering techniques [
To analyze the large volumes of data collected about the user environment, it is advisable to use
approaches based on data mining. The application of the data mining algorithms for solving the
problem of fraudulent transaction recognition and analysis of the user environment data is given in
[
The goal of this research is development of the infrastructure for collection and analysis of user environment as a part of anti-fraud system on the basis of big data processing technologies.
To achieve this goal, the following tasks were formulated:
Development of the structure of the system for collecting and analyzing information about the
user environment based on DM techniques.
Development of a structural and functional scheme of processing user environment data for
testing algorithms of detecting financial fraud on the basis of DM as a part of the system of
distributed processing of banking transaction data.
3. System structure for collecting and analyzing user environment information
Technologies of remote banking service (internet banking) for accessing accounts and operations
through a web browser do not require installation of the client part software and have become very
widespread [
Anti-fraud system evaluates risk of the current transaction and, in case of exceeding a certain
threshold value, triggers additional mechanisms for verifying the legitimacy of the transaction [
a phone call from a security specialist to the user.
In this architecture, user session management module (USMM) is the main element of the
antifraud system. The module analyzes the transaction data and the user environment data (UED)
collected by the client-side script. The implemented script forms a set of data about the user
environment [
The module’s main task is to classify the current session and its transaction (legitimate transaction or fraudulent actions) based on the composition of the analysis methods: signature and automatic. If the module's estimated legitimacy of the transaction is below the threshold, an additional mechanism for authenticating the user is triggered.
The signature analysis module allows the use an expert knowledge and their formalization in the form of a system of production rules ”IF-TO”. The main task of this module is to classify UED and/or existing data about banking transaction in order to detect fraudulent activities. A special feature of the module is the use of a unified signature database based on the system of production rules, which allows to integrate mechanism for explaining the decision in an understandable form for security expert into the system. Initially, the database of the signature analysis module contains typical templates of fraudulent and legitimate transactions and UEDs in appropriate cases. Replenishment of the signature database is possible both in manual mode through the interface of the manual analysis module, and with the help of the automatic analysis modules which detects signatures in the new data processing mode. Analysis of the accumulated data under control of the analyst makes it possible to identify new production rules in an automatic mode based on the DM technologies.
Task of the automatic analysis module is automatic data classification based on the DM methods,
for example, using a neural network classifier trained on a database of committed transactions marked
by an expert [
In the previous works [
The core of automatic analysis module is the neural network – the “black box” – which allows to assign the current vector of features extracted from the data collected about the user environment to one of the previously defined classes. The following classes are proposed: User system is under remote control; The user system uses action anonymization mechanisms; User system does not contain any suspicious elements.
Results of the signature and automatic analysis modules are comparing. If the verdicts do not coincide, then an anti-fraud system expert can be brought in for manual analysis of the precedent. While the accumulation of data on transactions and UED occurs, the neural network classifier is aftertrained. While group of use cases, the parameters of which do not fit into the current scheme of “IFTO” signature rules, are being formed, new rules are being extracted, replenishing the existing signature database. If new signatures are added by the expert, then the current base of marked use cases is analyzed in order to update the class labels and retrain the neural network classifier.
Manual analysis module is designed to correct the markup of the existing database of the user environment and transactions in order to form a training sample for the neural network classifier. The module allows expert to analyze the system solution for each of the use cases and correct it in case of erroneous operations.
The AFS control modules allow to monitor the basic performance of AFS, analyze the log of the system and debug the interaction of the signature and automatic modules.
Thus, structure of the system for collection and analysis of user environment as a part of anti-fraud
system is presented in figure 1. The key element of the system is data mining module. Analysis
algorithms should be applicable in the context of “Big Data” (a set of approaches, tools and methods
for processing structured and unstructured data of huge volumes and significant diversity for obtaining
human-readable results that are effective under conditions of continuous growth, distribution over
multiple nodes of the computer network) [
To date, there are many tools for the distributed processing of banking transaction data
(frameworks: Hadoop, Apache Spark, ClickHouse, ElasticSearch, Splunk Free) [
Main element of the distributed processing system of banking transaction data is a distributed file
system. Nowadays the most popular distributed file system is HDFS [
Next element of the big data processing system is the distributed programming and machine
learning infrastructure. The core of this element is Spark, a cluster computing infrastructure similar in
MapReduce [
To directly store the accumulated data, it is proposed to use solutions from the New-SQL family
[
A detailed description of the additional elements and their functions in the distributed data processing system for bank transactions is presented in the table 1. Operating system for big data application
Engine for big data processing Scikit-learn python library integrated in Apache
Spark for exploratory data analysis H2O library integrated in Apache Spark for
machine learning in Hadoop system TensorFlow library integrated in Apache Spark for
deep learning in Hadoop system Application for maintaining configuration
information, naming and etc Application for data summarization, SQL-like
query, and analysis Application for transferring data between relational
WorkFlow Monitoring
Flume [
databases and Hadoop Application for transferring data between relational
databases and Hadoop Application for collecting, aggregating, and
moving of unstructured data Web interface to monitor Hadoop system
The use of three machine learning libraries is due to the need of rapid prototyping of the developed algorithms, debugging on small amounts of data and the possibility of importing the developed models.
The hardware-software stand structure for testing the algorithms of detecting financial fraud based
on DM techniques in the distributed data processing system for bank transactions based on the
selected big data processing stack is shown on figure 3.
The data collection and analysis module consists of:
• Sentry software suite [
The load testing module is designed to automate collection of the user environment database. For distributed processing of bank transaction data, the Hadoop cluster is used, on which the software is deployed from Table 1. Typical configuration of the used server machine pool is shown in Table 2. ID 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Sector Collection and analysis user environment data Load testing software Hadoop Cluster
The main problem of improving the TMS efficiency is the insufficient amount of registered parameters transferred from an online banking client side to a processing center, and the imperfection of signature analysis methods and algorithms because of low adaptability and configuration flexibility.
Nowadays the most promising solution is the use of technologies for determining user environment in combination with the methods of machine learning within TMS. The use of machine learning is an indispensable criterion, because big amount of user environment information being collected and application of the rules to this data becomes difficult. Algorithms for analysis should be applicable in the context of “big data”.
In the paper the infrastructure for collection and analysis of user environment as a part of anti-fraud system on the basis of big data processing technologies is proposed. Acknowledgments Work was supported by the Russian Foundation for Basic Research, research № 17–48–020095.