The problems of big data clustering is very interesting area of artificial intelligence nowadays. This task often occurs in many application, that related with data mining, deep learning, web mining etc. For solving these problems the traditional approaches and methods require that every vector - observation from processed data set is fed in batch form and does not change over the time and could belong more, than one cluster. In this situation more effective are fuzzy clustering methods that are synthesized under the assumptions of mutual overlapping of classes, based on credibility theory and adaptive goal function. Therefore as alternative, to known clustering algorithms we propose adaptive recovery of distorted data algorithm based on credibilistic fuzzy clustering approach.
To solve a wide class of Data Mining problems, machine learning technologies are effectively used, among which the most complex ones are the ones based on selflearning approach and are used in conditions of a priori information shortage. These technologies, combined with a neural network approach, provide effective solutions to many real-world problems. Real systems of image processing and computer vision, control of aerospace objects, technical and medical diagnostics, in economics and finance, in military application, motion control, energy, forensic science, signal analysis of various nature, etc. have been created, and this list is expanded almost daily.
At the moment there is a sufficient amount of information about the activities of enterprises, hospitals, firms, which reflects the activities of these objects. After analyzing this information, it is possible to find objective patterns, provided that the table generated reflects actual data reflecting cause-effect relationships.
This information has been collected over the years (for example, the rate of inflation, the income level of the population, the structure of household spending, the cost of housing and communal services, the state of industrial and agricultural production, the timeliness of wages and pensions, etc.). Of course, such data is difficult to analyze manually due to the large amount of information and complex non-linear cause-and-effect relationships. Therefore, it became necessary to develop new methods of analysis and forecasting, which include machine-based methods for the detection of patterns.
In many data mining tasks related to the processing of empirical quantitative observations, the data may be distorted by omissions. The task of restoring such observations was given sufficient attention, while approaches based on machine learning, above all, artificial neural networks and fuzzy systems that solve the problem of recovering these lost observations are very effective in this situation.
At the same time, the described approaches to data recovery are workable only in cases when the
initial data are set a priori, and the “-porbojepcetrty” table or time series has a fixed number of
observations, i.e. do not change during processing. At the same time, there is a wide class of tasks,
when the data are received for processing sequentially, as it happens when learning Kohonen’s - self
organizing maps [
One of the problem of artificial intelligence is clustering of big data distorted by omissions. For work with such data artificial neural networks, neuro-fuzzy systems, hybrid systems known for their universal approximating properties and ability to learn, seem to be the most effective. For the normal functioning of a neural network or a hybrid system distorted data have to be restored in some way. At the same time, most of these algorithms process information in batch form, that makes it difficult to use them in cases when data for processing are fed sequentially of time series. In the tasks when vectors-observation are fed sequentially in online mode, the number of missing values is doesn't known in advance, and in this case known approaches are ineffective.
Therefore, as an alternative this approaches, methods and algorithms have been proposed for adaptive recovery distorted data based on credibilistic fuzzy clustering approach.
When solving real problems related to the processing of data obtained as a result of either experiments or observations, quite often there arises a situation when the vectors - observations x( ) contain missing observation that must be filled in the process before processing the source array. It is clear that such arrays with missing values can't be directly clustered, but must be preprocessed.
Let's present the original data set in the form of a traditional table, which contain information about N objects, each of than is described by (n 1) of feature-vector xT ( ) x1( ),..., xi ( ),..., xn ( ) . It is assumed that NG rows contain a missing values and NF N NG are full. ....
xN1 .... .... .... .... .... .... .... ....
p x1p xip .... .... x τp ....
xNp .... .... .... .... .... .... .... ....
j x1j ....
xij .... x τj .... xNj .... .... .... .... .... .... .... ....
n x1n xin .... .... x τn ....
xNn or 1 .... ....
i τ ....
N
That is, in fact, this table is the source array X T xi ( ),i 1, 2..., n; 1, 2...N in which there are absent NG
elements. Next, it is assumed that between the columns of the table xj xj (1), xj (2)...xj ( ),...xj (N )T there exists a linear correlation on the basis of which the missed values can be restored x j ( ) using a regression equation xˆ j ( ) aj0 aj1x1( ) a j2x2 ( ) a j, j1x j1( ) a j, j1x j1( )... a jn xn ( ) (1) xˆ j ( ) x j ( )a j (2) where a j (a j0 , a j1,...a jn )T (n 1) vector of parameters to be determined; x j ( ) 1, x1( ),...x j1( ), x j1( )... xn( ) - (1 n) - vector - features for -th object without j - th element and a unit in the first position.
Vector of unknown parameters a j can be found using the standard least squares method, for which from the matrix X T the -th row, j-th column are deleted and the column formed by units is added to the left. Next on the basis N1 n of matrix X Tj calculate the desired vector of estimates rows are removed and reduced on the basis of the (NF n) matrix n - times are vectors of parameters a j for all j 1, 2...n and missing values are filled with the received estimates.
This approach can be extended to situation when object data are fed to the matrix - table sequentially row by row. When the (N 1) -th observation in the form of a completely filled row xT (N 1) x1(N 1),..., xi (N 1),..., xn (N 1) vector-estimate a j can be used in addition to the recurrent method of least squares, or you can look at the online method of machine learning: a j (NF 1) a j (NF ) Pj (NF 1) Pj (NF )
Pj (NF ) xTj (N 1) x j (N 1)a j (NF )
1 xTj (N 1)Pj (NF )x j (N 1) Pj (NF )x j (N 1)xTj (N 1) Pj (N )
F , 1 xTj (N 1)Pj (NF )x j (N 1) x j (N 1), (3) after that, can be specified value xˆ j ( ) .
If (N 1) -th row contains missing values, it is skipped and the algorithm (3) waits row that contain all observation, for example x(N 2) , after that it is calculated a j (NF 1) by dint of (N 2) -th observations and all values are adjusted xˆ j ( ) , including xˆ j (N 1) .
At the initial stages of processing Table 1, when the number of completely filled rows NF is comparable to the number of columns n , the estimates obtained using the least squares method are characterized by low accuracy. In this situation, for sequential processing, it is more efficient to use adaptive learning algorithms that have both filtering and tracking (for non-stationary situations) properties.
The quality of solving various problems of forecasting, pattern recognition, inverse modeling, control, data recovery, etc. can be enhanced through the use of neural network ensembles, in which the same data are processed simultaneously by several n-parallel artificial neural networks. The output signals are combined in some way into an overall assessment, which gives an idea of the quality of the results obtained using the local networks of the ensembles, as shown in Figure 1.
The most widespread approach to uniting ANNs into ensembles are two ways, such as modular and based on weighted averaging. These approaches are quite different from each other, but they also have similarities. They both use a linear combination of the outputs of their members in one form or another.
x( )
y* ( ) ANN1 ANN2 ...
y1( ) y2 ( ) yh ( )
The modular approach has a rather heuristic character in contrast to a more mathematically rigorous weighted averaging, although here remains an element of subjectivity associated with the choice of members of the ensemble. This problem is usually solved with the help of certain heuristics, although there are more or less rigorous results based on genetic programming or a gradual increase in the complexity of the networks - members of the ensemble.
It is convenient to organize information processing in the sequential data arrival mode on the basis of a neural network system, the main elements of which are parallel adaptive linear associators (ALA), trained using formula (3). Figure 3 shows the scheme of this system, which does not require additional explanations.
The vast majority of known fuzzy clustering algorithms assume that the original data array X
contains N observation and do not change in the process of analysis. Can be noted, there exists a
fairly many Data Stream Mining tasks, where data are fed for processing sequentially and their
volume is a priori unknown, and Big Data Mining when this volume is so large that it simply does not
allow processing this data in batch mode. In such situations, recurrent fuzzy clustering algorithms
come to the fore, with the help of which these data are analyzed sequentially vector by vector as they
enter the system [
x1( ) x2 ( ) ... x j ( ) ... xn ( )
Line ... decoder containing x j ( ) spaces x1( ) x2 ( ) ... xn ( )
ALA1 ALA2
ALAn a1( ) a2 ( ) ... a j ( ) ... an ( ) xˆ1( ) xˆ2 ( ) ... xˆ j ( ) ... xˆn ( )
( n) - the table “objectproperty” ˆ X
Credibilistic fuzzy clustering algorithms are connected with the goal function [
N m E Credq ( ),cq Credq (k)D2 x( ),cq
1 q1 0 Credq ( ) 1q, , sup Credq ( ) 0,5 , Credq ( ) sup Credl ( ) 1 where Credq ( ) - level of observation x( ) credibility.
In the procedures of credibilistic fuzzy clustering, the level of membership is determined by the
membership functions [
It is easy to see that membership level (6) using the distance is based on similarity measure
[
q (k) q D x( ), cq q (k) 1
1 D2 x( ),cq . q ( ) 1 q ( ) q ( ) 1 D2 x( ), cq ,
supl ( ), Credq ( ) q ( ) 1 supl ( )
2 , cq 1
N Credq ( )x( )
N Credq ( ) 1 , (4) (5) (6) (7) (8)
Thus, if the fuzzy clustering algorithm in a batch form can be can be written as [
ll1q q ( 1) 1 x( q21()c1q)( ) 2 1 ,
1 q ( 1)
q ( 1) supl ( 1)
, 1 Credq ( 1) q ( 1) 1 supl ( 1),
2 c ( 1) cq ( ) ( 1)Credq2 ( 1) x( 1) cq ( ). q (9) (10)
It is easy to see that the recurrent fuzzy clustering algorithm is not more complex than the online modifications of probabilistic, possiblilistic, and robust procedures.
The proposed algorithm was tested in experimental research, that was conducted using well-known test data sets of the UCI repository: Abalone and Gas. The comparison results of mean error are demonstrated in Table 3. The mean error of the clusters centroids of proposed Adaptive Recovery of Distorted Data Based on Credibilistic Fuzzy Clustering Approach (AdCFCA) was compared with another well known methods of FCM and Gustafson-Kessel (GK). Adaptive Recovery of Distorted Data Based on Credibilistic
Fuzzy Clustering Approach
PC 0.51 0.26 0.24 Easy to see that the approach under consideration shows better results clustering quality.
To estimate the quality of Adaptive Recovery of Distorted Data Based on Credibilistic Fuzzy Clustering Approach, we used the overall accuracy comparison of 50, 100 and 150 experiments for datasets and another clustering procedures such as: FCM and Gustafson-Kessel (GK). AdCFCA 0.75 0.64 SC 1.63 1.65 0.64
On Figure 4 and Figure 5 the comparison of overall accuracy of algorithms was presented. Easy to see that the proposed approach is better than the known algorithms.
The problem fuzzy clustering of data distorted by missing values and outliers is considered. The recovery in the mode of sequential data arrival for processing is investigated. As an alternative to the classic possibilistic and probabilistic approaches we use of the essence modify credibilistic one in a case when distorted data are fed in online mode. The modification consists in the introducing to the clustering system additional recovery block of data that are processing in real time. For solving the clustering problem we introduce the adaptive gradient algorithm for minimizing of goal function, related with credibilistic fuzzy clustering and based on similarity measure of special type. Experimental research confirms the effectiveness of evolving approach. This algorithm is characterized by easy numerical implementation, high speed and can to process information in online mode when the data are fed sequentially in real time.
The work is supported by the state budget scientific research project of Kharkiv National University of Radio Electronics "Deep hybrid systems of computational intelligence for data stream mining and their fast learning" (state registration number 0119U001403).
[14] F. W. Young, R.M. Hamer Theory and Applications of Multidimensional Scaling-Hillsdale,
Erlbaum, N.J., 1994. [15] A. Shafronenko, Ye. Bodyanskiy, D. Rudenko, Online neuro fuzzy clustering of data with omissions and outliers based on completion strategy, in: Proceedings of The Second International Workshop on Computer Modeling and Intelligent Systems (CMIS-2019), Zaporizhzhia, 2019, pp. 18-27.