The paper is about research of the algorithms, methods of the classification and prediction objects' groups, depiction of the informationmathematical system, which is created on these algorithms' basis. They use variety methods of the machine learning and their compilations aggregative classifier, all of these is for the solution of the classification's problem, particularly borrower's solvency prediction. This helps to make previous preparation of the source data, which also contents discretisation, missed data's recovery and detection of the important factors for statistics, how to use these methods of the classification and create cogeneration models, how to analyze quality of these models using statistical measures, to predict objects' groups.
Nowadays there are a lot of models and methods for the solution the problem of prediction the class of objects. Next methods
was used for the analysis of credit risks’ assessment: decision trees [
Depends on concrete case every method of machine learning can be the best one from the side of prediction’s accuracy, so it
offers joint using of different classifiers, which are made on variety parts of learning sample [
To decide belonging borrower to one of the classes (creditworthy or not) on the basis of results of parallel application to t he original sample of certain methods of the classification, aggregation results is possible on three grounds: - by average value (the possibility of object belongs to class y 1 (“creditworthy client”) shall be considered as arithmetical average of belonging probabilities of object to class y 1, which were found out using all nine methods of classification); - by median (first of all, expansion is ranging, which contains results base methods of classification in the combination, probability is counted through calculation result of average classifier in the case of their odd number or in the case of eve n number probability is counted through half-sum of results of average classifier);
- by voting (result of the aggregative classifier in this case is average result of classification’s basic methods, which gave fact of the belonging object to y 1 class with ≥ 0,1 probability).
There is a solution algorithm for the assessment of clients’ creditworthiness on the basis of aggregative classifies, it contains next stages:
Data Science / V.А. Alekseeva, Yu.E. Kuvayskova
Formation and processing of original sample. This stage consists in dividing sample into learning one (for making
classification models) and test one (for checking accuracy of the made models), recover missed data [
Information-mathematical system of credit score was made in the basis of listed above algorithm. It allows predict the class of the object (for example, borrowers’ creditworthiness) using learning sample. Software package was devised in the programming support environment Matlab R2014a, which contains all of methods initial data computing and amount of algorithms machine learning, which are needed for solvation the classification problem. Initial data is information about clients, which is personal details and relevant class of the creditworthiness “old” clients; personal details of “new” clients; personal details, credit history and credit transaction terms and conditions of borrowers, who repay a loan.
Program allows making previous preparation of initial data: recover missed information; characteristics’ discretisation; coding nonnumeric data; selection statistically worthy characteristics. All of listed above classification methods instantiates in the program, which includes aggregative classifier with the possibility of selecting criteria of aggregation (by average value, by median, by voting).
Method of L-fold cross-check is used for making classifiers for getting unbiased estimator of quality parameter. The essence of this method is in division original sample to L non-crossing parts, which are approximately equals to each other by the extent. It is possible to choose L’s value, it varies from 3 to 10. In turn every part serves as test sample, rest ones aggregates to learning sample. Summative assessment of the classifier’s quality is defined by averaging mistakes in all L test sample. It allows exclude possibility of fudge to the best prediction.
In conclusion constitutes values of quality of created models for three cutoff thresholds (cutoff threshold – value, which if target is higher than the target become the one from class y 1 ): cutoff threshold 0,5; definitive cutoff threshold; custom cutoff threshold. Definitive classification threshold is the least deviation between mistakes of I-class and II-class.
Quality control of created classification models and aggregative classifiers makes with helping of next characteristics [
Customer can estimate which method or method combination gives the best result for objects and make prediction for original set of characteristic values using specified criteria. Working process of aggregative classifier is making by program, so optimal method combination is formed automatically using special criteria, after this customer can differ compare results of aggregative classifier and basic classification methods.
As the first example there are results of program working on realization aggregative classifier for sample of German bank’s clients, which includes 900 borrowers, who have 20 characteristics (status of current checking account, credit history, loan purpose, credit length, loan proceeds, average balance on the savings account, work experience in the last place, income in %, family status, guarantors, permanent residence in the last place, data on property, age, available loans, type of housing, number of previous loans in this bank, type of activity, number of dependents, phone availability, citizenship), and one dependent b inary variable (borrower is creditworthy and non-creditworthy). This program provides previous data processing, including characteristics’ discretisation and coding nonnumeric data, such as citizenship of client, education, family status etc., with numbers. Nine different classification methods and aggregative classifier are analyzed. Aggregation was made by average value. It is possible to make aggregation using all of three characteristics. A 10-fold cross-check was used in this classification.
For target sample is got optimal aggregative classifier with 0,5 cutoff threshold, which contains next methods: neural nets, logit regression, bagging decision trees, created function method, fuzzy inference models. There is results of program in tab.1. The best classification result is got with helping of aggregative classifier, because of mean-root error of aggregative classifier is less than other methods; the highest percent of right prediction of creditworthy clients is in two methods: aggregative classifier and bagging decision trees, but I-class error of aggregative classifier is lower; aggregative classifier gives average value for prediction for non-creditworthy clients, but with minimal II-class error.
There are just three levels of quality of classifiers in this table. Also, program allows form diagrams, which show areas und er
ROC curves (AUC). Fig.1 shows such diagram for target sample. ROC curve [
The paper [
Researching of data about creditworthiness of Australian borrowers was made similar. Names of variables and their values were coded for Privacy Policy. Data includes one of dependent binary variable, which means creditworthiness (takes value 0 in case of non-creditworthy client or 1 in case of creditworthy client) and 14 independent characteristics. There are 690 examinations.
Optimal aggregative classifier for target sample was found 0,5 with cutoff threshold, which contains next methods: neural nets, logit regression, Bayesian classifier and fuzzy inference models. Results of working are in the Table 2. The best result was made with aggregative classifier.
Data Science / V.А. Alekseeva, Yu.E. Kuvayskova
These examples show possibilities of this information-mathematical system of credit score. Method is selected from all of possible methods and it allows predict creditworthiness or non-creditworthiness of clients at the same time, minimizing meanroot error and I-class, II-class errors and maximizing AUC level. Using current method it is possible to find in which class is target using specified set of values. Also, this program allows renovate models if there is new data.
It considered using nine known methods of machine learning and their combinations for solvation the problem of binary classification of objects. It is not possible to explain effectiveness just one of the methods, because for different samples, even for different parts of one sample, is possible to get variety results. These methods and algorithm of making aggregative classifiers are realized in terms of information-mathematical system of credit score.
This program allows find the best model or optimal aggregative classifier, Classifier was the best one for targets samples. In the case with German borrowers the most accurate prediction was received by using combination of next methods: neural nets, logit regression, bagging decision trees, created function method, fuzzy inference models; classifier includes neural nets, logit regression, Bayesian classifier and fuzzy inference models in case with Australian data. Aggregative classifier helps to get the purpose – increase of prediction accuracy of creditworthiness clients of the bank.
This system of the credit score can be used for any problem of binary classification, for prediction of technical condition of
objects [