Customs duties are based on the origin and value of the goods and their classification (the customs tarif to be applied). Falsifying any of these factors when importing or exporting products is fraud. This includes falsely declaring the origin of the goods, declaring a lower value on the goods, misclassifying the goods and smuggling goods. In this paper we apply machine learning algorithms (Artificial Neural Network, Fuzzy Min-Max Classifier and Logistic Regression) for fraud detection in customs declarations. Performance of the models are evaluated using accuracy, sensitivity and specificity. The best results were achieved using Logistic Regression. In further research it would be useful to analyze applicability of ensemble learning methods and others fraud detection models.
Customs fraud is any fraudulent attempt to reduce the customs duty (ex.: tax) imposed on
goods when they are imported to particular country from abroad [
An analysis of the literature has shown that research has been mostly carried out in the areas of healthcare and financial crime. Study [2] analyzed credit card fraud. The sample for this study consisted of information on 978 cases of fraud and 22 million legal transactions. Decision tree algorithms (CART, C5.0, and CHAID) and support vector machines (SVMs) with diferent kernel functions were used to detect fraud cases: polynomial, sigmoidal, and linear. The results of the analysis revealed that more accurate results are obtained using decision tree algorithms.
The support vector method was also compared with logistic regression models [3]. The results showed that the regression model has more accurate results (with accuracy of 91 %) than the SVM classifier (with accuracy of 82 %).
In order to develop a methodology to detect fraudulent financial statements of companies, a data set consisting of 76 companies engaged in manufacturing was analyzed [4], from which 38 of these firms were accused of falsifying financial statements. Few methods were compared experimentally: neural networks, decision trees, and Bayesian networks. In this case, the most accurate results were obtained using Bayesian networks (with accuracy of 90.3 %), followed by the 80 % accuracy of neural networks and 73.6 % of decision trees.
All of the above methods belong to the class of supervised learning methodology and are used to create models based on historical data of fraud cases. In this paper we are also going to use these techniques.
Feature selection is the process of reducing the number of input variables when developing a classification model. It is recommended to reduce the number of input variables to reduce the computational cost and to improve the performance of the model. Statistical based feature selection methods involve evaluating the relationship or diference between each input variable and the target variable using statistical tests. The choice of statistical measures depends on the data type of both the input and output variables.
Feature selection between numerical input variables and categorical output variable: 1. The t-test is used to determine if the means of two samples are significantly diferent from each other. Suppose that ( 1, 2, … , ) and ( 1, 2, … , ) data samples are obtained by measuring two independent normally distributed random variables ∼ ( , 2 ) and ∼ ( , 2 ), when the means , and the variances 2 , 2 are unknown. Testing statistical hypothesis [5]: { 0 ∶ = , the two populations are similar 1 ∶ ≠ , the two populations are diferent. (1) Normality condition of the two independent samples can be checked using Shapiro-Wilk test or the central limit theorem says no matter what distribution things have, the sampling distribution tends to be normal if the sample is large enough ( > 30) [5]. The independent samples t-test comes in two diferent forms: the standard Student’s ttest, which assumes that the variance of the two groups are equal ( 2 = 2 ) and the Welch’s t-test which do not assume that the variance is the same in the two groups. The F-test is applied to test the hypothesis of the equality of two independent samples variances. the diference between two variables.
a) Cohen’s for the Student t-test [5, 6]: 2. After performing the t-test, Cohen’s efect size is calculated to measure the strength of = −
, where and represent the mean values of the two samples, is an estimator of the pooled standard deviation of the two samples. It can be calculated as follow: b) Cohen’s for the Welch’s t-test[6, 5]:
where 2 and 2 are the variance of the two samples, and are sample sizes.
= √ ( − 1) 2 + ( − 1) 2 + − 2
, = √
− ( 2 + 2)/2 .
T-test conventional efect sizes, proposed by Cohen, are: 0.2 (small efect), 0.5 (moderate efect) and 0.8 (large efect) [6]. Variables that have a moderate or large efect size are selected for classification model developing.
Feature selection between categorical input variables and categorical output variable: categorical variables [5]. Testing statistical hypothesis: 1. The 2 test of independence is used to test whether there is a relationship between two ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎩
the two categorical variables. ⎪⎧⎪ 0 ∶the variables are independent, there is no relationship between
between the two categorical variables. ⎪⎪ 1 ∶the variables are dependent, there is a relationship ⎪ of the relationship between two variables [7]: frequency < 1 the Fisher’s exact test is used.
The 2 test of independence works by comparing the observed frequencies to the expected frequencies if more than 20% of expected frequencies < 5 and at least one expected 2. After performing the 2 test, Cramer’s efect size is calculated to measure the strength = √
2 ⋅ where is the sample size,
= min( − 1, − 1) and is the the number of rows and efect size depends on the degrees of freedom , shown in Table 1. is the number of columns in the contingency table. The interpretation of Cramér’s (2) (3) (4) (5) (6)
In practice, unbalanced data sets often occur, which can cause major problems in classifying with machine learning algorithms. The class imbalance afect process of machine learning, algorithm completely ignores the minority class [8]. This happens because machine learning algorithms are usually constructed to improve accuracy by reducing the error. To solve this problem, class balancing is performed in two ways: Synthetic Minority Oversampling Technique (SMOTE) and a combination of Random Over-Sampling and Random Under-Sampling techniques.
Random Over-Sampling method aims to increase the number of instances in the minority class by randomly duplicate examples in the minority class. Random Under-Sampling method aims to balance class distribution by randomly delete or merge examples in the majority class. This process is done until the majority and minority class instances are balanced out [9].
SMOTE is an over-sampling method in which the minority class is over-sampled by creating new synthetic examples rather than by over-sampling with duplication. The minority class is over-sampled by taking each minority class sample and introducing synthetic examples along the line segments joining any / all of the minority class nearest neighbors [10].
Artificial Neural Networks are a component of artificial intelligence that is based on the functioning of a human brain and consists of units of computation called nodes or neurons. The neuron multiplies the input by the corresponding weights, then applies an activation function to the weighted sum and creates an output. To express this mathematically we can describe it by equation:
( ) = ( ⊙ + ), here – input vector, – weight vector, – threshold value, ⊙ – the product of the corresponding elements, – activation function, – output. Neuron functioning is depicted in Figure 1. (7)
The Fuzzy Min-Max Classifier that we are using in this paper is based on fuzzy logic. Suppose
we have a dimensional model of ℎ and membership functions. membership functions
are sets of fuzzy logic that define input participation in a particular class. Therefore, the input
defined by the set of fuzzy logic will belong to the class to which the model gives the highest
degree of membership [11, 12]. Membership degree can be defined mathematically by equation:
= {,
( )},
(8)
here – is a space of objects whose common element is marked as . Fuzzy set (class)
in space is defined by membership function ∼ ( ) which connects each point in the space
X with a real number in the interval [
The architecture of this network is similar to the architecture of a simple neural network depicted in Figure 1. The first layer is the − input layer. In a hidden layer, each node is defined in the and dimensional space and belongs to a particular class. The last layer consists of class nodes, which quantity is equal to the number of possible classes. It is this layer that is diferent from a simple neural network. Here the outputs in the layer are used as membership functions, and the class node that provides the highest estimate resolves the value of the input class [11, 12].
Logistic Regression is used when the dependent variable is categorical. From our example: to predict whether a fraud is not detected (0) or (1). Logistic regression is valid under these general assumptions: independent variables do not have to be normal, normally distributed errors are not required, and the homoscedasticity of the dependent variable is not examined. However, this model has the disadvantage of being sensitive to the problem of multicollinearity. To avoid this, independent variables need to be chosen so that none of them are linear variables of the rest.
After selecting the required variables the probability of the event = 1 is calculated: ̂ ( = 1|⃗) =
̂ + ̂ 1 1+ ̂ 2 2+…+ ̂ 1 + ̂ + ̂ 1 1+ ̂ 2 2+…+ ̂ where ⃗ is a vector of independent variables. After this calculation a threshold can be selected and therefore class selected.
The following evaluation metrics were used to evaluate the performance of potential fraud detection models: accuracy, sensitivity, specificity [13]. These measures can be calculated based on the confusion matrix, which is a table with two rows and two columns that reports the number of false positives, false negatives, true positives, and true negatives (Figure 2).
The confusion matrix itself is relatively simple to understand, but the related terminology can be confusing [13]: 1. TP and TN indicate that cases of fraud and normal activity are correctly classified (predicted). 2. FP means that normal activity was misclassified as a fraud. 3. FN indicates that the fraud was misclassified as a normal activity.
Accuracy is the proportion of true results (both true positives and true negatives) and total number of cases [13]:
+ = .
+ + +
Sensitivity of a classifier is the ratio between correctly identified positives and actual positives [13]: (10) = +
(9) (11)
Specificity of a classifier is the ratio between correctly classified negatives and actual negatives [13]: =
The sample of the survey consists of anonymized declarations submitted to customs, which can be 15 types, the types of declarations are given in Table 2. The period for submission of declarations is 9 days. The received declaration in the system is evaluated according to the defined rules, the output of the rules system is the label low risk of fraud, medium risk of fraud or high risk of fraud. After inspection by a customs oficer, the declaration is marked F if fraud was not detected and T if fraud was detected. Except declarations with reliable customs estimates were selected for the research, i.e. from low risk of fraud class with rating T, from medium risk of fraud class with rating T and from high risk of fraud class with rating F and T. Total number of declarations in the data set is 992. In addition to the main 306 attributes that describe the data, there are attributes-flags that describe what procedures were performed. Depending on the type of declaration, only certain attribute fields are filled in, consequently there are many omitted values in the data set. More detailed information about the data set and its attributes cannot be provided due to security requirements. 5. Experiments 5.1. Data preprocessing and feature selection 1. Due to the large number of attributes (877) and the small sample size (992 declarations), attributes with more than 50 % missing values, attributes with equal values, text attributes and categorical variables that have more than 30 categories were removed. After this preprocessing, 104 attributes remained in the data set, of which 78 are factors, 24 are numeric, identification number and label (F or T). 2. Between each numeric input variable and the target variable Cohen’s efect size for t-test was calculated and 11 of the 24 variables that had a moderate or large efect size were selected for classification model developing. Also between each categorical input variable and the target variable Cramer’s efect size for 2 test was calculated and 42 of the 78 variables that had a moderate or large efect size were selected for classification model developing. 3. In the train-test split procedure 75 % of the data set records were randomly assigned to the train data set and 25 % to the test data set. 4. Data set balancing was performed in two ways: Synthetic Minority Oversampling Technique (SMOTE) and a combination of Random Over-Sampling and Random Under-Sampling techniques. 5. Categorical variables were expressed in numerical binary expression using one hot encoding technique and min-max normalization was used for numeric variables.
The experiment results are presented at Table 3. It was found that the best results in terms of accuracy are obtained using Logistic Regression and Fuzzy Min-Max Classifier methods, when the training set is balanced according to the combination of over-sampling and under-sampling (accuracy is 0.92 and 0.89, respectively) and Logistic Regression with 0.92 accuracy when the data is balanced by SMOTE method. After evaluating the results according to the sensitivity index, the most appropriate methods are logistic regression and the Fuzzy Min-Max Classifier (indicators equal to 0.96 and 0.94, respectively). In terms of specificity, the highest values of the indicator were achieved using Logistic Regression and the Fuzzy Min-Max Classifier with data balanced by the SMOTE method (results equal to 0.56 and 0.53, respectively).
From the compiled classifications, the Logistic Regression model trained with a combination of balanced training data reached the highest values of all indicators. With this model, declarations without irregularities (F) are identified with 96 percent accuracy, and possible cases of fraud (T) with 56 percent accuracy. The application of fuzzy logic in the construction of the neural network improved the values of the assessment indicators compared to the neural network when no fuzzy logic was applied in the combination method with balanced data and at least improved the value of the specificity indicator only from 0.34 to 0.35 when training with SMOTE balanced methods. However, a neural network based on fuzzy logic is not the most efective method for identifying possible cases of fraud, as models constructed by the Logistic Regression method provide more accurate classification results.
Experiments with data set of customs declarations show that: 1. The best results in terms of accuracy are obtained using Logistic Regression and Fuzzy Min-Max Classifier models, when the training set is balanced according to the the combination of oversampling and undersampling (accuracy is 0.92 and 0.89, respectively) and Logistic Regression model with 0.92 accuracy when the data is balanced by SMOTE method. According to the sensitivity, the most appropriate methods are Logistic Regression and the Fuzzy Min-Max Classifier (indicators equal to 0.96 and 0.94, respectively). In terms of specificity, the highest values were achieved using Logistic Regression and the Fuzzy Min-Max Classifier with data balanced by the SMOTE method (results equal to 0.56 and 0.53, respectively). 2. From the compiled classifications, the Logistic Regression model trained with a combination of balanced training data reached the highest values of all evaluation metrics. The application of fuzzy logic in the construction of the neural network improved the values of the assessment indicators compared to the neural network when no fuzzy logic was applied in the combination method with balanced data. 3. Identification methods of fraud, constructed as classifiers for labeled customs declarations data could be more efective if the data set were larger and contained more examples of fraud.
In further research it would be useful to analyze variety of ensemble learning methods and others fraud detection models. [2] Y. Sahin, E. Duman, Detecting credit card fraud by decision trees and support vector machines, IMECS 2011 - International MultiConference of Engineers and Computer Scientists 2011 1 (2011) 442–447. [3] S. Y. Huang, Fraud detection model by using support vector machine techniques, 2013. [4] E. Kirkos, C. Spathis, Y. Manolopoulos, Data mining techniques for the detection of fraudulent financial statements, Expert systems with applications 32 (2007) 995–1003. [5] V. Čekanavičius, G. Murauskas, Statistika ir jos taikymai, Vilnius: teV 1 (2000). [6] J. Cohen, Statistical power analysis for the behavioral sciences, 2nd edn. á/l, 1988. [7] H.-Y. Kim, Statistical notes for clinical researchers: Chi-squared test and fisher’s exact test, Restorative dentistry & endodontics 42 (2017) 152–155. [8] N. Japkowicz, S. Stephen, The class imbalance problem: A systematic study, Intelligent
Data Analysis (2002) 429–449. [9] G. E. A. P. A. Batista, R. Prati, M. C. Monard, A study of the behavior of several methods for balancing machine learning training data, SIGKDD Explorations 6 (2004) 20–29. [10] N. V. Chawla, K. W. Bowyer, L. O. Hall, W. P. Kegelmeyer, Smote: Synthetic minority over-sampling technique, J. Artif. Int. Res. 16 (2002) 321–357. [11] P. Gajjewar, Understanding fuzzy neural network using code and animation, 2018. URL: https://medium.com/@apbetahouse45/ understanding-fuzzy-neural-network-with-code-and-graphs-263d1091d773. [12] P. K. Simpson, Fuzzy min—max neural networks—part 1: Classification, IEEE Trans. on
Neural Networks 3 (1992) 776–786. [13] A. Tharwat, Classification assessment methods, Applied Computing and Informatics (2018). URL: http://www.sciencedirect.com/science/article/pii/S2210832718301546. doi:https://doi.org/10.1016/j.aci.2018.08.003.