Introduction

Clustering is a technique in machine learning and data analysis that involves grouping a set of data points into subsets, or clusters, based on the similarity between them. Fuzzy clustering is a variation of traditional clustering methods that allows for more flexible and nuanced assignments of data points to clusters [1][2][3][4][5]. In contrast, fuzzy clustering allows data points to belong to multiple clusters simultaneously with varying degrees of membership. This reflects the inherent uncertainty or ambiguity present in real-world data.

The Fuzzy C-Means (FCM) algorithm, proposed by James Bezdek in 1973, is a prominent method in fuzzy clustering [6]. FCM assigns membership degrees to data points, indicating the likelihood of each point belonging to different clusters. This flexibility makes fuzzy clustering particularly useful in scenarios where data points may exhibit overlapping characteristics or uncertainty in their categorization.

Over the years, fuzzy clustering has found applications in diverse fields, including pattern recognition, image processing, and Data Mining. Researchers have developed various extensions and enhancements to the original FCM algorithm, addressing specific challenges and improving its adaptability to different data patterns.

The validity of fuzzy clustering solutions became a key focus, leading to the introduction of indices to assess the quality of clustering results. These indices help researchers and practitioners evaluate the effectiveness of fuzzy clustering algorithms in capturing meaningful patterns within datasets.

The evolution of fuzzy clustering has seen ongoing advancements, with researchers exploring sophisticated membership functions and integrating fuzzy clustering with other machine learning techniques. This integration has expanded the capabilities of fuzzy clustering, making it applicable to complex problems in large-scale data analysis.

The era of big data has significantly influenced the field of clustering, including traditional clustering methods and the development of fuzzy clustering techniques. Deep learning in big data represents a powerful combination that has transformed various fields by enabling more sophisticated analysis, pattern recognition, and decision-making capabilities.

Recently, there has been significant research into leveraging deep learning to uncover meaningful data representations through neural networks. A notable area of exploration involves the integration of unsupervised clustering algorithms with stack neural networks. This synergy has become a vibrant field of research, aiming to jointly optimize the performance of deep learning models and clustering algorithms.

The goal of the work is propose the stack neuro-fuzzy system for Data Stream Mining using credibilistic approach and designed to work both in batch and its recurrent online version.

Neural Network Data Compression

An important problem that arises when processing large amounts of observations is data compression to highlight the most essential information and identify certain latent factors that implicitly determine the nature of the phenomenon being studied. One of the most effective approaches to solving this problem is the apparatus of factor analysis [7], which has found wide application in problems of processing empirical data in various fields: psychology, sociology, technology, economics, medicine, criminology, etc.

The basic idea of factor analysis, which allows for the presence of a priori unknown hidden factors, leads to the following informal task: by observing a large number of measured parameters (indicators), identify a small number of parameter-factors that mainly determine the behavior of the measured parameters, or in other words: knowing the values of a large number of measured functions parameters, set the appropriate values of the factor arguments common to all functions and restore the form of these functions.

The initial information for factor analysis is the ( )

N n

× observation matrix:

( )

1 2 1 2 1 2 1 2(1) (1)

(1) ...

T n T n T n T n x x x x x x x x X N x k x k x k x k x N x N x N x N                   = =                      (1)

that formed by an array of N n-th dimensional vectors ( )

1 2 ( ) , ,..., T N x k x x x =

and autocorrelation matrix ( )

n n × ( )( ) 1 1 1 1 ( ) ( ) ( ) ( ) ( ) ( ) ( ), N N T T k k R N x k x N x k x N x k x k N N = = = − − = ∑ ∑   (2)

where

1 1 ( ) ( ), ( ) ( ) ( ) N k x k x k x k x k x k N = = = − ∑ (3)

vectors of measured indicators centered relative to the average of the data array. One of the most common and effective methods for finding factors is the principal component method or component analysis, which is widely used in problems of data compression, pattern recognition, coding, image processing, spectral analysis, etc. and known in pattern recognition theory as the Karhunen-Loeve transform.

The task of component analysis is to project data vectors from the original n-dimensional space into a m-dimensional one ( ) m n < space of principal components and reduces to searching for a system λ , etc. In other words, the problem comes down to finding solutions to the matrix equation:

( )

( ) 0 j n j R N I w λ − = such, that 1 2 ... 0 m λ λ λ ε ≥ ≥ ≥ ≥ ≥ and 1 j w = .

The dimension of the space of principal components m is determined, as a rule, from empirical considerations and the required degree of compression of the data array.

Thus, in algebraic terms, solving a factor problem is closely related to the problem of eigenvalues and finding the rank of the correlation matrix; in a geometric sense, this is the problem of moving to a lower-dimensional space with minimal loss of information; in a statistical sense, this is the problem of finding a set of orthonormal vectors in the input space that "accept" the maximum possible variation of the data, and finally, in an algorithmic sense, this is the problem of sequentially determining a set of eigenvectors 1 2 , ,..., m w w w by optimizing a set of local criteria that form a global objective function (

)

2 1 1 1 1 1 ( ) m m k k k T j j j j p E E w x p k k = = = = = ∑ ∑∑  with constraints 0, ,1.

T T j l j j w w j l w w =≠ =

The first principal component 1 w can be found by maximizing the criterion:

( )

2 1 1 1 1 ( ) k k T p E w x p k = = ∑  (4)

by solving a nonlinear programming problem using uncertain Lagrange multipliers.

However, if data processing must be carried out in real time, neural network technologies come to the fore, among which the self-learning rule and E. Oya's neuron should be noted.

It is with the help of Oya's rule in the form:

( )

1 1 1 1 1 1 1 1 ( 1) ( ) ( ) ( ) ( ) ( ) ( ) , ( ) ( ) ( ), (0) 0 T w k w k k y k x k w k y k y k w k x k w η  + = + −   = ≠     (5)

the first principal component can be isolated.

Next, following the procedure of standard principal component analysis, from each vector ( ), 1, 2,...,

x k k N = 

its projection onto the first principal component is subtracted and the first principal component of the differences is calculated, which is the second principal component of the original data and the orthonormal first. The third principal component is calculated by projecting each original vector ( )

x k  into the first two components, subtracting this projection from ( )

x k  and finding the first principal component of the differences, which is the third principal component of the original data array. The remaining principal components are calculated recursively according to the described strategy.

It is this idea of recursive calculation of principal components that forms the basis of the algorithm proposed by T. Sanger [8] and in a modified form having the form [9]

 + = +  = −   = ≠   = =   = = − + ≤ ≤     (6)

It is easy to see that the first principal component is calculated using the Oya algorithm, then the projection of the input vectors onto 1 ( ) w k are subtracted from the inputs and the differences are processed by the next neuron, etc. In Fig. 1 shows a diagram of a modified artificial T. Sanger's neural network, composed of E. Oya's neurons and implementing the algorithm (6).

The first layer of the network is formed by encoder neurons that pre-process signals by centering and normalizing them. Further signals 1 2 ( ), ( ),..., ( )

n x k x k x k   

are processed in the second hidden layer formed by E. Oya's neurons, after which they are sent to the output layer formed by elements with activation rectifier functions with a dead zone , , ( ) 0,

u if u u otherwise θ ψ ≥  =  (7)

which allows you to highlight informative signals ( )

j

y k and filter out the noise.

The Sanger neural network is an effective means of compressing information with minimal loss of accuracy, but its capabilities are limited by the fact that, implementing essentially the standard technique of factor analysis, it solves a linear problem, while the main advantage of neural network technologies is the ability to work in purely nonlinear situations.

The problem of nonlinear factor analysis can be effectively solved using credibilistic theory and clustering analysis.

Fuzzy credibilistic clustering

Alternatively, to probabilistic and possibilistic procedures [10] it was introduced credibilistic fuzzy clustering approach using as its basis the credibility theory [11] and is largely devoid of the drawbacks of known methods.

The most common approach within the framework of probabilistic fuzzy clustering is associated with minimizing the goal function [12][13][14].

( ) ( ) 2 1 1 ( ), ( ) ( ), N m q q q q k q E u k c u k d x k c β = = = ∑∑(8)

with constraints

1 1 ( ) 1, 0 ( ) . m q q N q k u k u k N = =  =     < <   ∑ ∑ (9)

Solution of nonlinear programming problem using the method of Lagrange indefinite multipliers leads to the well-known result [9,11,15]:

( ) ( ) ( ) ( ) ( ) ( ) 1 2 1 1 1 2 1 1 1 ( ),( ) , ( ), ( ) ( ) ( )q q m l l N q k q N q k d x k c u k d x k c u k x k c u k β β β β − − = = =   =        =     ∑ ∑ ∑ (10)

coinciding with 2 β = a popular method of Fuzzy C-Means of J. Bezdek (FCM) [6]. If the data are fed to processing sequentially, the solution of the nonlinear programming problem ( 8), ( 9) using the Arrow-Hurwitz-Uzawa algorithm leads to an online procedure:

( ) ( ) ( ) ( ) ( ) 1 2 1 1 2 1 1 ( 1), ( )( 1)

, ( 1), ( )

( 1) ( ) ( 1) ( 1) ( 1) ( ) . q q m l l q q q d x k c k u k d x k c k c k c k k u k x k c k β β β η − − =  +  + =    +   + = + + + + −   ∑ (11)

The goal function of credibilistic fuzzy clustering has the form [6,11] close to (8)

( ) ( ) 2 1 1 ( ), ( ) ( ), N m q q q q k q E Cred k c Cred k d x k c β = = = ∑∑(12)

with "softer" than ( 9) constraints: 0 ( ) 1, for all and , sup ( ) 0,5, for all , ( ) sup ( ) 1, for any and ,for which ( ) 0,5.

q q q l q Cred k q k Cred k k Cred k Cred k q k Cred k ≤ ≤   ≥   + =   ≥  (13)

It should be noted that the goal functions ( 8) and ( 12) are similar and that there are no rigid probabilistic constraints in (13) on the sum of the membership in (9).

In the procedures of credibilistic clustering, there is also the concept of fuzzy membership, which is calculated using the neighborhood function of the form:

( ) ( ) ( ) ( ), q q q u k d x k c ϕ = (14)

monotonically decreasing on the interval [0, ] ∞ so that (0) 1, ( ) 0.

q q ϕ ϕ = ∞ →

Such a function is essentially an empirical similarity measure of [13,15,16] related to distance by the relation:

( ) 2 1 ( ) . 1 ( ), q q u k d x k c = +(15)

Note also that earlier it was shown in [14] that the first relation (10) for 2 β = can be rewritten as ( )

1 2 2 ( ), ( ) 1 , q q q d x k c u k σ −     = +    (16)

where ( )

1 2 2 1 ( ), m q l l l d x k c ε σ − = ≠     =       ∑ (17)

which is a generalization of the function (15) (for 2 1 q σ = (15) coincides with (17)) and satisfies all the conditions for (14).

In batch form the algorithm of credibilistic fuzzy clustering in the accepted notation can be written as

q q q q l q q l l q N q k q N q k u k d x k c u k u k u k Cred k u k u k Cred k x k c Cred k β β − − * * * ≠ = =  = +   =      = + −          =     ∑ ∑ (18)( ) ( ) ( ) ( ) ( ) 1 2 1 1 1 ( ) 1 ( ), , ( ) ( ) sup ( ) , 1 ( ) ( ) 1 sup ( ) , 2 ( ) ( ) ( )

and in the online mode, considering ( 16), ( 17):

( ) ( ) ( ) 2 2 1 1 2 2 1 ( 1) ,( 1), ( ) ( 1), ( ) ( 1) 1 , ( 1) ( 1) ( 1) , sup ( 1) 1 ( 1)

( 1) 1 sup ( ) , 2

q m l l l q q q q q q l q q l l q

q q q q k d x k c k d x k c k u k k u k u k u k Cred k u k u k c k c k k Cred k x k c k β σ σ η = ≠ − * * * ≠  + =  +       +    + = +    +     + + = +   + = + + −     + = + + + + − ∑         (19)

From the point of view of computational implementation, algorithm (19) is not more complicated than procedure (11) and, in the general case, is its generalization to the case of credibilistic approach to fuzzy clustering.

Experimental research

Conducting experimental studies and comparative analysis of the quality of data clustering using various metrics allows you to objectively assess the effectiveness of the developed method in accordance with analogues. To estimate the quality of the method we used quality criteria partitioning into clusters such as [3,6] Separation Index (S): on the contrary of partition index (SC), the separation index uses a minimum-distance separation for partition validity.

Xie and Beni's Index (XB): it aims to quantify the ratio of the total variation within clusters and the separation of clusters. The optimal number of clusters should minimize the value of the index.

Dunn's Index (DI): this index is originally proposed to use at the identification of "compact and well separated clusters".

So the result of the clustering has to be recalculated as it was a hard partition method. The specific information of the data sets is shown in Table 1.

Discussions

Upon analyzing the results acquired, it can be inferred that irrespective of the volume of the initial data provided, the processing through the proposed method exhibits comparable speed and clustering quality when contrasted with established clustering algorithms and methodologies.

The obtained results confirm that the performance stack neuro-fuzzy system is better than other network structures, and it can be a viable structure for Data Stream Mining.

The results of accuracy that demonstrated in Table 4 confirm that proposed method online stack fuzzy credibilistic clustering for Data Stream Mining time more superiority regardless of the number observations that are fed on clustering process.

Based on the experimental findings, it is advisable to endorse the proposed system for practical application in addressing the challenges associated with automatic clustering of large datasets.

Conclusion

The problem of fuzzy clustering of data streams by stack neuro-fuzzy system is considered. In the paper was proposed the stack neuro-fuzzy system for Data Stream Mining using credibilistic approach and designed to work both in batch and its recurrent online version.

The network shows that stack structures based on fuzzy models can be applicable in data clustering. The proposed stack neuro-fuzzy system is quite simple in numerical implementation and can use the well-known online fuzzy clustering methods intended for solving Data Stream Mining problems.

Future research endeavors could explore the potential of employing stack neuro-fuzzy systems for fuzzy clustering of data streams, aiming to address the complexities inherent in automatic clustering of big data.