Introduction

The current state of technological development is inextricably linked with the development of computerized tools, which, in turn, are dependent on the mathematical apparatus and practical algorithms that use it. The development of computer tools, in particular hardware, acts as a catalyst for the development of existing and the emergence of new scientific fields, such as Data Science. Modern capabilities of computing environments allow the implementation of algorithmically sufficiently complex methods that are the basis of intellectual analysis. And this should become an impetus for the development of new hardware and software systems based on the theoretical principles of artificial intelligence.

Recently, in the tasks of analyzing and processing non-stationary signals of an arbitrary nature under conditions of uncertainty, computational intelligence methods are increasingly being used, among which hybrid neural networks can be distinguished.

By the task of data segmentation, we will understand the division of the data sample into homogeneous homomorphic segments based on the analysis of changes in the internal properties of the data. Currently, several segmentation methods are known, namely using wavelet analysis [1], fractal-wavelet technologies [2], neuro-fuzzy technologies [3][4][5], etc.

Depending on the specifics of the problem being solved, two main types of forecasting and segmentation methods can be applied: real-time and batch.

Many neural network architectures, including hybrid structures, are used to solve this kind of problems, but these systems are either cumbersome in their architecture or not sufficiently adapted for real-time learning. In most cases, the activation functions of such networks are sigmoidal functions, splines, polynomials, and radial basis functions.

Credibilistic fuzzy clustering

Traditionally, the initial information for the clustering problem is a sample of observations consisting of N n -dimensional feature vectors:

{ }

(1), (2),..., ( ),..., ( ) ,

X x x x k x N = ( ) 1 ( ) ( ),..., ( ) , T n n x k x k x k R = ∈ k = 1, 2, …, N,

and the result of the algorithm is the distribution of the initial data set into m classes with a certain level wj(k) belonging to the k-th feature vector of the j-th cluster. At the same time, there is a wide class of problems when the initial information comes not in vector, but in matrix form, i.e.

1 2 ( ) { ( )}; i i x k x k = where i1 = 1, 2, …, n1, i2 = 1, 2, …, n2, k = 1, 2, …, N.

Such situation is characteristic, for example, in image processing [6], when the initial (N1 × N2)-matrix is divided into N = N1N2(n1n2) -1 (n1 × n2) fragment matrices that are subject to clustering, because of which the homogeneous in some sense segments of this image. Traditionally, this problem is solved by preliminary vectorization of fragments and the use of already known procedures, the most popular of which is the method of clustering fuzzy C-means [6,7].

To process matrix data, it is necessary to introduce matrix methods of clusteringsegmentation, for which it is advisable to introduce into consideration the matrix method of fuzzy C -means, which is a generalization of FCM. This method will avoid unnecessary vectorizationdevectorization operations when processing data given in the form of two-dimensional arrays and provides information processing in online mode.

So, let the sample of observations be given

1 2 1 2 ( ) { ( )} , n n i i x k x k R × = ∈ 1, 2,..., k N = ,

at the same time, for the convenience of further processing, these data are pre-centered relative to the average:

1 1 ( ) N k x x k N = = ∑ (1)

and normalized to its spherical norm (Frobenius norm):

( )

( ) ( ) ( ) . T x k Tr x k x k =(2)

The matrix probabilistic criterion is used as the objective function of clustering:

( )

2 1 1 1 1 ( ( ), ) ( ) ( ( ), ) ( ) ( ( ) )( ( ) ) , N m N m T j j j j j j j k j k j E w k c w k D x k c w k Tr x k c x k c β β = = = = = = − − ∑∑ ∑∑(3)

in presence of constraints

1 1 1 ( ) 1 ( ) 1 0, 1, 2,..., , 0 ( ) , 1, 2,..., . m m m j j j j j j w k w k k N w k N j m = = = = ∨ − = = < < = ∑ ∑ ∑

By introducing the Lagrange function:

2 1 1 1 1 2 1 1 1 ( ( ), , ( )) ( ) ( ( ), ) ( ) ( ) 1 ( ) ( ( ), ) ( ) ( ) 1 , N m N m j j j j j k j k j N m m j j j k j j L w k c k w k D x k c k w k w k D x k c k w k β = = = = β = = =   λ = + λ − =         = + λ −           ∑∑ ∑ ∑ ∑ ∑ ∑(4)

where λ(k) -uncertain Lagrange multiplier, and solving the system of Kuhn-Tucker equations:

1 2 1 1 ( ( ), , ( )) ( ) ( ( ), ) ( ) 0; ( ) ( ( ), , ( )) ( ) 1 0; ( ) ( ( ), , ( )) 2 ( )( ( ) ) O, ( ) j j j j j m j j j j j N j j j j k j L w k c k w k D x k c k w k L w k c k w k k L w k c k w k x k c c k β− = β =  ∂ λ  = β + λ = ∂   ∂ λ  = − =  ∂λ     ∂ λ    = − − =    ∂      ∑ ∑ where ( ( ), ,( ))

( )

j j j L w k c k c k   ∂ λ     ∂     - 1 2 ( ) n n × -matrix formed from partial derivatives1 2

( ( ), , ( )) ;

j j ji i L w k c k c ∂ λ ∂ Ο

-matrix of the same dimension formed by zeros, thus, we arrive at the final form of the algorithm:

1 2 1 1 2 1 1 1 1 2 1 1 1 1 ( ( ( ), )) ( ) ; (( ( ), )) ( ) ( ( ), ) ;

( ) ( ) . ( )

j j m l l m l l N j k j N j k D x k c w k D x k c k D x k c w k x k c w k −β −β = −β −β = β = β =   =             λ = − β                =     ∑ ∑ ∑ ∑ (5)

The resulting system gives rise to a wide class of clustering procedures. Thus, if we set β = 2, we get a simple and effective matrix clustering algorithm [8], which is a generalization of the popular procedure of J. Bezdek [6]:

1 1 1 2 1 2 1 ( ( ( ) )( ( ) ) ) ( ) ; ( ( ( ) )( ( ) ) ) ( ) ( ) , ( ) T j j j m T l l l N j k j N j k Tr x k c x k c w k Tr x k c x k c w k x k c u k − − = = =  − − =   − −      =    ∑ ∑ ∑ (6)

where Tr -matrix trace symbol. The main difference between probabilistic and possibilistic approaches is that probabilistic algorithms use relative similarities between objects and clusters, while probabilistic algorithms use absolute similarities.

Instead of the fuzzy partition matrix in the fuzzy C-means algorithm, the possible C-means algorithm uses a ( ) N m × -matrix of possibilities or typicality matrix T = {tj(k)}, where tj(k) ∈ [0, 1] -the possibility that the object x(k) belongs to cluster j.

The possibilistic matrix has only the following limitations:

1 0 ( ) , 1, 2, , . m j j t k m k N = < ≤ = … ∑ (7)

This means that an object can have a feature vector that contains only values close to zero (usually such objects are considered noise) or only ones.

Krishnapuram, Keller et al proposed the probabilistic C-means (PCM) algorithm and two algorithms that combine probabilistic and possibilistic approaches: the probabilistic-possibilistic C-means algorithm (FPCM) and the possibilistic-probabilistic C-means algorithm (PFCM) [9][10][11].

In the PCM algorithm, formula (6) was replaced by the expression:

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 1 1 1 1 1 ; 1 , j T j j j N j k j N j k t k Tr x k c x k c t k x k c t k β− β = β = =   − −          +   γ         = ∑ ∑ (8)

where γj > 0 -a constant determined empirically. It can be seen that the calculation of the cluster prototype in formulas ( 6) and ( 8) is identical, with the only difference that the matrix of fuzzy partitioning is changed to the matrix of possibilities. The calculation of the possibility of an object belonging to a cluster in formula ( 8) can be justified as a bell-shaped function presented in Figure 1.

( ) ( ) ( ) ( ) ( ) ( ) 1 1 T j j j N k N j k j k Tr x k c x k c K w w k = β = β − − γ = ∑ ∑ ,(9)

where K > 0 (most often K = 1). But calculations γj by formula (9) requires memory to store the fuzzy partition matrix, as well as time for its use.

The PCM algorithm does a good job of suppressing interference and can usually be applied when it is necessary to improve the results obtained with the help of other algorithms. Also, this algorithm can merge close clusters into one, from which it follows that the initial number of clusters that was set in advance is too large (at the same time, the PCM algorithm can merge clusters that should be separated).

The FPCM and PFCM algorithms use both a fuzzy partition matrix and a feature matrix, trying to take advantage of both approaches.

The FPCM algorithm uses the following formulas:

( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

( )

1 1 1 1 1 1 1 1 1 1 1 1 ; ; ( ) , T j j j m T l T j j j N T j j l N j j k j N j l j k l Tr x k c x k c k Tr x k c x k c Tr x k c x k c t k Tr x c x c w k t k x k c w l l w k t k −β −β = −η −η = β η = β η =  − −   =  − −     − −  =   − −    +   =  +   ∑ ∑ ∑ ∑ (10)

where η > 0 (in most cases η = 2). The FPCM algorithm uses the standard procedure for calculating the fuzzy partition matrix, but the possibility matrix is calculated using a new formula. Cluster prototypes are calculated using the sum of both matrices.

The PFCM method uses a standard procedure for calculating the fuzzy partition matrix (as in formula ( 6)). The procedure for calculating the possibility matrix was taken from PCM (8) and slightly modified. Centroids are calculated as in the FPCM algorithm, but both matrices have their own weights:

( )( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

( )

1 1 1 1 1 1 11 1 ; 1 ; 1 ( ) ,−β −β = β− β η = β η =  − −   =  − −      =     − −    +    γ      +  =   +   ∑ ∑ ∑ (11)

where a > 0, b > 0. The constants a and b determine the relative importance of the fuzzy partition matrix and the capability matrix in the centroid calculation function. By setting a = 0, algorithm (11) goes to PCM, and by setting b = 0, algorithm (11) goes to FCM.

Analyzing all the presented methods, several conclusions can be drawn. First, the membership function of the FCM algorithm with its limitations is too "strong", allowing outlier objects to be assigned to one or more clusters, which, in turn, can greatly affect the underlying structure of the data set. On the other hand, the PCM method's constraint on the features is too weak -it allows to refer to a cluster independently of the rest of the data. Also, PCM is very sensitive to the initialization of the capability matrix. The PFCM method is an efficient combination of the two approaches, and the clustering results depend on the parameter setting a, b, β, η.

Algorithm ( 6) can be extended to the case when data for processing are received sequentially in on-line mode. To do this, by applying the Arrow-Hurwitz-Uzawa saddle point search procedure to the Lagrangian (4), when the (k+1) th observation is received, the estimates of the membership levels and centroids can be refined using recurrence relations [12] 1), ( )) ) ( 1) ;

1 2 1 1 2 1 1 ( ( (

( ( ( 1), ( )) )

( ( 1), , ( 1)

) ( 1) ( ) ( ) ( ) ( ) ( 1)( ( 1) ( )), j j m l l j j j j j j j j D x k c k w k D x k c k L w k c k c k c k k c c k k w k x k c k −β −β = β  +  + =   +       ∂ + λ +    + = − η =    ∂       = + η + + −  ∑ (12)

for an arbitrary value of the fuzzifier β and

,

T j j j m T l l l j j j j Tr x k c k x k c k w k Tr x k c k x k c k c k c k k w k x k c k − − =  + − + − + =   + − + −    + = + η + + −  ∑ (13) for β = 2.

It is easy to see that the expression ( 13) is an adaptive version of the procedure (4), and ( 13) is, accordingly, (6).

The matrix credibility criterion is used as the objective function of clustering:

( ) ( ) ( )2 1 1 1 1 ( ), ( ) ( ), ( ) ( ( ) )( ( )) N m N m T j j j j j j j k j k j E Cred k c Cred k D x k c Cred k Tr x k c x k c β β = = = = = = − − ∑∑ ∑∑(14)

in presence of constraints

0 ( ) 1 , ,sup ( ) 0,5 , ( ) sup ( ) 1 jj j l Cred k j k Cred k k Cred k Cred k  ≤ ≤ ∀  ≥ ∀   + = (15)

where ( )

j

Cred k -level of observation ( ) x k credibility. In the procedures of credibilistic fuzzy clustering, the level of membership is determined by the membership functions [13]:

( )( ) ( ) ( ) ( ), ( ( ) )( ( )) T j j j j j j w k D x k c Tr x k c x k c ϕ ϕ = = − −(16)

where j ϕ -decreases monotonically on the interval [0, ] ∞ and with condition (0) 1, ( ) 0

j j ϕ ϕ = ∞ → .

It is easy to see that membership level (16) using the distance is based on similarity measure [14]. As such a measure in [15], it was proposed to use a function:

( )1 ( ) . 1 ( ( ) )( ( )) j T j j w k Tr x k c x k c = + − −(17)

Thus, if the fuzzy clustering algorithm in a batch form can be written as [16]:

( )

1 1 1 ( ) , 1 ( ( ) )( ( ) ) ( ) ( ) , sup ( ) ( ) 1 sup ( ) ( ) ,2( ) ( ) , ( ) j T j j j j l j l j N j k j N j k w k Tr x k c x k c w k w k w k w k w k Cred k Cred k x k c Cred k β β * * * = =  =  + − −    =    + −  =     =     ∑ ∑ (18)

in the online mode this procedure (18) has the form (19):

( ) ( ) ( ) ( ) 1 1 2 1 1 1 1 2 ( 1) (( 1) )( ( 1) ) , ( ( 1)

)( ( 1)

) ( 1) 1 ,( 1)

( 1) ( 1) , sup ( 1)

1 ( 1) ( 1) 1 sup ( 1) , 2 ( 1) ( )( 1)  + = + − + −       + − + −   + = +   +     + + = + + = + + − + + = + + ∑ ( ) ( 1) ( 1) ( ) j j d k x k c k β                   + + −  (19)

or in case when

2 β = ( ) ( ) ( ) 1 2 2 1 1 2 2 ( 1) ( ( 1) )( ( 1) ) , (( 1)

)( ( 1)

) ( 1) 1 ,( 1)

( 1) ( 1) , sup ( 1)

1 ( 1) ( 1) 1 sup ( 1) , 2 ( 1) ( )( 1)

m T j j j l l j T j j q j q j l j j l

j j j k Tr x k c x k c Tr x k c x k c w k k w k w k w k Cred k w k w k c k c k k Cred σ σ η − − = ≠ − * * *     + = + − + −         + − + −   + = +   +   + + = + + = + + − + + = + + ∑ ( ) ( 1) ( 1) ( ) . j k x k c k                   + + −  (20)

It is easy to see that the recurrent fuzzy clustering algorithm is not more complex than the online modifications of probabilistic, possibilistic, and robust procedures [17,18].

Experimental research

Digital images, including satellite images of the city of Kharkiv, were used to test the implemented matrix credibilistic modifications of the clustering algorithm. Samples have no missing attributes and are numeric.

The result of the algorithm is the final fuzzy partition matrix for all sample objects and class prototypes.

When processing digital images, objects (matrices or vectors of the same dimensions) are formed from fragments of this image, and each pixel from the RGB (Red-Green-Blue) color model is converted to the Grayscale model, where the brightness of a pixel is expressed as a scalar value from the interval [0,1]. The conversion from the RGB model to the Grayscale model is performed according to the formula:

(0.299 0.587 0.114 ) / 255 Y R G B = + + ,

where Y is the brightness of the pixel glow, R, G, B are the brightness of the glow of red, green, and blue tones, respectively, the values of which are in the interval [0, 255]. Observation sets formed from digital images are processed according to the same principle as standard quantitative samples. After image processing, each cluster is assigned the colors of the Grayscale model, and each object is colored in the color of the nearest cluster.

To evaluate the quality of the algorithm, the following criteria were used: Partition Coefficient (PC), Classification Entropy (CE), Partition Index (PI) with the same initialized fuzzy partition matrix U0.

Table 1 shows the results of the accuracy and speed of the clustering algorithms on the Iris sample, and Table 2 shows the results of the satellite digital image of the city of Kharkiv. The time given is an average for one iteration, considering the vectorization-devectorization operation. On Figure 2 show the initial image, the resampled sample (20% of objects) and the result of the cluster analysis and the process of the algorithm.