Fuzzy Clustering of Biomedical Datasets Using BSB- Neuro-Fuzzy-Model

Abstract

A special neural networks that contain autoassociative memory (AM) - BSB- and GBSB-models are investigated at this paper. These models are implemented on hypercube and solve the task of dataset clusterization due to the fact of point attraction properties of hypercube peaks. A BSB-neuro-fuzzy model can be based on BSB-model as well due to the introduction of the special fuzzy membership function. A training algorithm for the BSB- neuro-fuzzy model is proposed. This algorithm enables to enrich the BSB-neuro-fuzzy model by adaptive properties. An experiment based on medical datasets proved a high quality of the proposed model.

Keywords

• fuzzy clustering
• hypercube
• attractor
• adaptive learning algorithm
• stable states

-

One of important properties of human brain is a property of information storage and its recovering using association system. Any images ever saw by person can be recovered after long time even in the case of its changing. These brain properties can be simulated by neural networks of associative memory (NN_AM)[ 1-6 ].

This artificial memory can be presented by direct-driven neural network (called static associative memory) and by recurrent neural network (called dynamic associative memory), those during its training can stack all patterns (memorizing phase). In recall phase of its functioning dynamic associative memory can make association new proposed pattern with all ever saws. Having said so all patterns ever proposed to the associative memory compose a fundamental memory set.

The basic difference of neural networks of associative memory from approximating neural networks (ANN) consist of the fact that ANN realize nonlinear mapping when neural networks of associative memory form mapping of all possible input vectors x in y(k) . Input vector x belong to neighbor x(k) such that

x  x(k)   , where y(k) – (m 1) fundamental memory vector, x(k) – (n 1) fundamental memory vector, k  1, 2, 3,..., l – a total number of fundamental memory pattern,  – a special positive parameter.

We are exploring the modified special class of neural network of associative memory is investigated. This memory implements mapping

for all x belonging to a neighbor area that can be described by  parameter. The main goal of associative networks is recovering of damaged information or information presented by partial pieces, for example in the area of medical diagnostics when the data fed into processing with gaps and outliers. 2

BSB-neuro model

“Brain-State-In-a-Box Model” was described by D. Anderson with colleagues [ 7,8 ]. This model is one of simple and effective architecture amount the structures of associative memory neural network [ 9-16 ] and it has the serious theoretical justification.

BSB-model is a neurodynamic nonlinear feedback system with amplitude constraint with the positive feedback. A dynamics of this system can be described in the state space using equation (1) where x(k, 0)  x(k) – input vector-image;   0,1, 2,...,  – iteration of machine time; x(k, ) – state vector in steady mode;  – small positive parameter of feedback connection; W – (n  n) – matrix of synaptic weights for correlation AM presented by one-layer neural network formed by adaptive linear associators;  () – activation piecewise linear function with saturation acting to elements of vector y(k, ) component-wise like

Therefore, the phase space of BSB-model is limited by n -dimensional hypercube whose center is in grid origin and which edge has a length equal two. Whole hypercube has 2n corners, which should be numbered. For this purpose it is useful to replace negative coordinates by zeros, and after that to change the obtained binary value to the decimal form adding a unit to it. Having said so to corner with all negative coordinates (-1,-1,…,-1) correspond 1-st number and to corner with all positive coordinates (1,1,…,1) – 2n number. 3

BSB-neuro-fuzzy-model (2) (3) 1, if  xi(k,  1)   ( yi(k, ))  yi1(,k, ),

if i  1, 2,..., n.  yi (i, )  1, yi (k, )  1, if

1  yi(k, )  1, n where d(x(k, ), xq )   xi(k, )  xq,i – Hamming distance between x(k, ) and i1 hypercube corner xq, q=1,2,…,2n. It is easy to see that  p (x(k, ))  1 , and membership level for most long-distance corner from x*p is equal to zero.

It makes a sense to find the connection between fuzzy clustering based on BSBmodel and the most popular fuzzy c-means algorithm (FCM) [ 17 ]. In FCM-algorithm the membership level x(k, ) to q-th corner-centroid of cluster can be defined as

The BSB-model solves the task of clusterization of input dataset x(k) , k  1, 2,..., l being in same time an AM. All hypercube corners proceed like pointed attractors with a significant domain of attraction to divide all n -dimensional feature space. At this situation a problem connected to capacity W of AM is appearing. The capacity W cannot exceed n (absolute capacity l / n  1) when the number of hypercube corners equals 2n  n . It is lead to two possible situations: at first, many corners will be «empty» and, at second, data belong to the same cluster can be placed in closely-spaced corners. That’s why it is rational to add a special neighborhood function between hypercube corners and consider a pattern from closely-spaced corners belonging to one cluster.

It will be useful to employ ideas of fuzzy clustering [ 17-21 ] to apply like neighborhood function the most simple triangle activation function. The membership level of pattern x(k, ) to q -th corner can be defined as q(x(k, ))  1  d(x(k, ), xq ) ,

2n q(x(k, ))  d 1(x(k, ), xq ) 2n  d 1(x(k, ), xl ) l1 , (4) n where d(x(k, ), xq )   xi(k, )  xq,i – Hamming distance between pattern i1 x(k, ) and hypercube corner xq, q=1,2,…,2n. It interesting to remark that because the pattern x(k, ) belongs to one of hypercube corners, for example, x p , Hamming distance between x p and xq can be defined as double number mismatched coordinate signs, that correspond to these corners.

Equation for  p(x(k, )) (4) can be transformed to the form q(x(k, ))  d 1(x(k, ), xq )

2n d 1(x(k, ), xq )   d 1(x(k, ), xl ) l1 lq   1 

1 d(x(k, ), xq ) 2n  d 1(x(k, ), xl ) l1 lq 1  1 

1 d(x(k, ), xq ) where  q – a width parameter of bell-shaped membership function of pattern x(k, ) to corner xq . It is easy to see that if x(k, )  xq , membership level is identical equal to one too. The form of membership function for different numbers n of input feature vector x(k) is presenting on Fig.1. is (n  l) ). (k  l) and X(1)  x(1) , excepting X  X(l).

Equation (5) can be rewrited in recurrent form: where X  X(l)  (x(1), x(2),..., x(l)) – fundamental memory matrix (matrix size In the following we will use also matrixes X(k)  (x(1), x(2), x(3),..., x(k)) (5) (6) This capacity depends on tuning procedure of n2 synaptic weights in adaptive linear associators. As such simplest procedure the D. Andersson assessment [ 7,8 ] can be used like:

It is easy to see that for previously centered and normalized vector x(k) the equation (5) describes autocorrelation matrix on pattern sequence and the expression (6) is a learning Hebb’s rule in standard form, widely used in neural networks applications.

For reducing an influence of disturbance component and for improving a quality of recovering we need to minimize errors of recovering  (r) that means making an orthogonal projection of pattern to fundamental memory vectors. A solving can be obtained after minimizing the criterion or, it is the same, minimizing the spherical norm

For these tasks it’s expedient to employ the linear projective adaptive algorithms, especially the most widespread autoassociative Widrow-Hoff rule that minimizes criterion where (k) is a scalar training rate value which can be selected empirically. An optimization for a timing this rule leads to a procedure (7) (8) that can be named is general version of S. Kaczmarz algorithm [ 22,23 ] on multidimensional case.

that minimizes the energetic function 5

GBSB-neuro-model

Nowadays different modifications along with the standard BSB-model (1) are used widely. Among them we may mark out a model [ 4 ] where 0    1 – forgetting factor,  – a small positive parameter, providing a permanent presence in the model of a pattern x(k)  x(k, 0) , that was stored. This modification of BSB-model has high convergence speed and fault tolerance.

At [ 13,15-16 ] Generalized Brain-State-in-a-Box Model (GBSB-model) was introduced. The synaptic weights matrix in this model is nonsymmetrical. The dissymmetry can appear after using for Kaczmarz-Widrow-Hoff algorithm and it leads to misconvergence to minimum of adopted energetic function. From other point of view symmetric properties of W accumulate «negatives» of fundamental memory patterns [ 13 ] in BSB-model that forms false attractors.

To prevent this disadvantage it is possible to introduce the GBSB-model, using expression:

where g  (n 1) – a vector, added in (9) for removing false attractors. 6

Experimental results

To investigate BSB-models functioning on medical dataset we have selected a dataset that consists of 182 patterns (patients), each of them is characterizes by 24 features. This dataset describes a psychophysiological human state needed to learn excitative and inhibitory processes on human body. All patients was divided on 2 classes: humans with predominance of excitative processes and ones who are prone to inhibitory processes. All data previously were normalized and centered to be occurred to hyn percube 1; 1 and the class feature was eliminated from dataset [ 24 ].

All datasets were transmitted to processing on BSB-model and 82% of pattern occur to 2 different corners of hypercube, when other 18% occur to the nearest ones. We have used the membership function (4) to refine the class type of these patterns. Then we have compared the result with known class type and obtained the clusterization accuracy about 92%. The comparison BSB-model with k-means algorithm shows the comparable accuracy of those approaches (about 85% for k-means). The fuzzy c-means algorithm can not be used for a comparison because we need to make the features compression before its clusterization (because of norm effect concentration). 7

Conclusion

The issue of synthesizing adaptive training algorithms for a special type of AM based on BSB- and GBSB-neuro-fuzzy models is considered. The introduced recursive procedures have a high speed, and fuzzy membership functions allows to relate the reconstruction process in the neural network model to the fuzzy-clustering procedures. This approach permits to expands the functionality of the developed method. The practical problem of partitioning into groups (clustering) of factors determining the predominance of excitation/inhibition processes in the body is solved.

References

1. Graupe , D. : Principles of Artificial Neural Networks (Advanced Series in Circuits and Systems) . Singapore: World Scient. Publ. Co. Pte . Ltd., ( 2007 ).

2. Rojas , R.: Neural Networks . A Systematic Introduction . Berlin: Springer-Verlag ( 1996 ).

3. Hagan , M.T. , Demuth , H.B. , Beale , M. : Neural Network Design. Boston: PWS Publishing Company, ( 1996 ).

4. Haykin , S.: Neural Networks and Learning Machine . New York: Prentice Hall, Inc. ( 2009 ).

5. K .-L., Du , & M. Swami , Neural Networks and Statistical Learning . London: Springer-Verlag ( 2014 ).

6. Bodyanskiy Yevgeniy , Perova Iryna, Fast medical diagnostics using autoassociative neurofuzzy memory // Int . J. of Computing, 16 ( 1 ), 2017 , 34 - 40 .

7. Anderson , J. A. : Cognitive and psychological computation with neural models . IEEE Trans. on Systems, Man, and Cybernetics , ( 13 ) 799 - 815 ( 1983 ).

8. Anderson , J.A. , Silverstein , J.W. , Ritz , S.A. , Jones , R.S.: Distinctive features. Categorical perception and probability learning: Some applications of a neural model . In Anderson , J.A. , Rosenfeld , E. (eds) “Neurocomputing: Foundations of Research”. Cambridge, MA: MIT Press, 413 - 451 ( 1988 ).

9. K. Abe , F. Sevrani , On the Synthesis of Brain-State-in-a-Box Neural Models with Application to Associative Memory . Journal Neural Computation , Vol. 12 ( 2 ), 451 - 472 ( 2000 ).

10. Yen , G. , Michel , A.N.: A learning and forgetting algorithm in associative memories: Results involving pseudoinverses . IEEE Tr. on Circuits System , ( 38 ) 1193 - 1205 ( 1991 ).

11. Zak , S.H. , Hui , S. , Dynamical analysis of the Brain-State-in-a-Box (BSB) neural models . IEEE Trans. on Neural Networks, #3 , 86 - 94 ( 1992 ).

12. Anderson , J.A. : The BSB model: A simple nonlinear autoassociative neural network . In Hassoun M.H. (eds) “ Associative neural memories: theory and implementation ” N.Y.: Oxford University Press, 77 - 103 ( 1993 ).

13. Lillo , W. E. , Miller , D.C. , Hui , S. , Zak , S.H. : Sythesis of Brain-State-in-a-Box (BSB) based associative memories . IEEE Transactions on Neural Networks , ( 5 ) 730 - 732 ( 1994 ).

14. Hui , S. , Bohner , M. Brain State in a Convex Body . IEEE Tr. on Neural Networks , ( 6 ) 1053 - 1060 ( 1995 ).

15. Park , J. , Cho , H. , Park , D. Design of GBSB neural associative memories using semidefinite programming IEEE Transactions on Neural Networks , # 10 ( 4 ), 946 - 950 ( 1999 ).

16. Hui , S. , Lillo , W. E. , Zak , S.H. , , Learning and forgetting in generalized Brain-State-in-aBox (BSB) neural associative memories . Neural Networks , # 9 ( 5 ), 845 - 854 ( 1996 ).

17. Bezdek , J.C. : Pattern Recognition with Fuzzy Objective Function Algorithms . N.Y.: Plenum Press, ( 1981 ).

18. Kruse , R. , Borgelt , C. , Klawonn , F. , Moewes , C. , Steinbrecher , M. , Held , P. Computational Intelligence . Methodological Introduction. Berlin: Springer-Verlag, ( 2013 ).

19. Kacprzyk , J. , Pedrycz , W., Eds., Springer Handbook of Computational Intelligence . Springer-Verlag Berlin Heidelberg, ( 2015 ).

20. L. C. Jain , C.L. Mumford Computational Intelligence , Berlin: Springer-Verlag, ( 2009 ).

21. Iryna Pliss , Iryna Perova Deep hybrid System of Computational Intelligence with Architecture Adaptation for Medical Fuzzy Diagnostics . I.J.I.S.A. , # 7 , 12 - 21 , ( 2017 ).

22. Kaczmarz , S. : Angenaeherte Ausloesung von Systemen linearer Gleichungen . Bull. Int. Acad. Polon. Sci. , (Let .A) 355 - 357 ( 1937 ).

23. S. Kaczmarz Approximate solution of linear equations . International Journal Control , # 53 , 1269 - 1271 , ( 1993 ).

24. Yevgeniy Bodyanskiy , Iryna Perova, Adaptive human machine interaction approach for feature selection-extraction task in medical data mining . Int. Journal of Computing , 17 ( 2 ), 113 - 119 ( 2018 ).