This paper describes research of artificial neural networks based on adaptive resonance theory. Discovered the shortcomings and problems of applying the existing structures of these networks in real-time systems. To solve the problem, a new model of the recognition field F2 of the ART-2 network is proposed, which is a tree structure, with a recurrently changing similarity parameter for each subsequent level. At each level, the similarity parameter increases, that leads to a sequential search for an active resonating neuron. Computational experiments have been carried out to prove the effectiveness of the proposed approach in comparison with the classical realization of networks of adaptive resonance theory.
Nowadays, more and more tasks of automated and automatic recognition of an object technical
conditions are solved using intelligent methods of data analysis [
ANN can be used to recognize images represented by both graphic forms and numeric parameters.
To recognize patterns represented by numerical parameters, dominating network described by
Grossberg and Carpenter [
Fig. 1 shows the structure of ART-2. It consists of three main fields F1, F2 and G [
Theorems that describe main principles of ART networks are proved in [
There are no fundamental differences in learning and recognition processes with such structure of a
network [
The algorithm of the network working consists of the following stages.
Stage 1. The input of S-layer receives a vector of signals consisting of n elements. Neurons of Wlayer perceive the signals of S-layer and add them to the output signals of the U-layer. wi si aui ;
i 1..n.
Stage 2. Output signals wi of W-layer neurons proceed to the inputs of X-layer and into the normalizing element WN, which calculates the norm: w n wi2 ; i1 where e – small positive constant that prevents division by zero.
Stage 3. The output signals of V-layer are determined by expression: vi f (xi ) bf (qi ); where f() – threshold function, to suppress noise signals: where – noise threshold.
Stage 4. Output signals vi of V-layer neurons proceed to the inputs of U-layer and into the normalizing element VN, which calculates the norm:
i 1..n.
Then repeat stages 3-5. Stable state in F1-field is set after two iterations.
Stage 6. The output signals from the P-layer proceed to the recognition layer Y, and the maximum element yi.is calculated then an output signal from the Y-layer is calculated:
n y j zij pi ;
i1 j 1..m; i max max(Y ); pi ui zi max id; The normalizing element RN calculates an output signal.
ri e u c p .
(1) (2)
Stage 8. If r , (where – the parameter of the correspondence of the expected result to the current one, which changes in the interval [0;1]), then it is assumed that the currently active neuron of Y-layer with an index imax of F2-field, is the winner. Then there is additional training of its weights. u zi max i d (1 d ) i zi max i ;
1 d r n ri2 .
i1 i 1..n.
If r , then the current active neuron of Y-layer is frozen and not involved in further competition.
Stage 9. If not all neurons Y-layer are frozen then proceed to Stage 6, otherwise a new neuron is created in the Y-layer, m increases by 1. It is considered that this neuron will resonate with new image, so weights are calculated as follows: zmi d (1 d ) ui .
1 d
Figure 2 shows a graph of the change in the weight value z1 of the resonant neuron of the Y-layer, with a fixed approximate output signal in the F1-field (U1≈0,62).
This graph shows that the changes of weights in the memory of F2-field aims to a limit, and it
confirms theorem formulated in [
When using the classical implementation of the network the following problems have been identified.
1. Significant increase in recognition time when learning a network with a large number of different data vectors with a great similarity parameter, since for each class of images a separate neuron is created, in the Y-layer. 2. The impossibility of parallelization of the recognition process, in view of the fact that the memory, which is represented by the matrix of weight coefficients z, is sensitive to the input sequence of recognizable images. 3. Long search for the active Y-layer neuron, since the search is performed only based on the maximum yi (1), which characterizes the frequency of input images.
Consider the proposed structure of ANN ART-2 with a modified memory (ART-2m). It consists of three main fields F1, F2, G. Field F1 consists of the same elements as in the classical implementation ART-2, as described in Section 1 above. (3)
As follows from the theory of adaptive resonance, after checking all neurons of the Y-layer new element ym is added. Verification of the correspondence of the rising signal to the active descending signal from the Y-layer occurs as follows:
This statement and the theorem, experimentally confirmed above, allow us to organize a multilevel memory with different parameters on each of the levels. Thus, the F2-field can be represented as layers of recognition Y-neurons with different levels detailing of the images and M-layers – semantic neurons connecting Y different levels, F2 = {Y, M}. Then, the G-field is a tuple consisting of control neurons Ri (where i = 1..n, n – number of input parameters), RN is a normalizing element, Riterp – threshold element ( p 1..kc. , kc – number of memory levels), G = {R, Riter, RN}. Fig. 3 shows the structure of the fields F2 and G with modified memory.
To reduce the search time of an active neuron yi, F2-field structure can be represented as a tree of memory with a concrete measure of similarity at each of the levels of the tree Riter, moreover, the learning of memory at the same level does not affect the weight of the above elements.
The algorithm of the network consists of the following steps.
Stages 1-5. Working in the F1-field, there in no differences from the classical implementation described above.
Stage 6. Required to calculate Riter for each level. If the difference between the similarity parameter and r
will be smaller, then тем more number of different neurons of Y-layer will be created, Accordingly, it is required to construct a recurrence relation of finding the similarity parameter for each level of the memory tree. In this research used the following recurrence relation with the initial measure of similarity equal to 0.5:
Riter1 0.5; Riterk 1 Riterk 0.75(1 Riterk );
i 1..kc.
Stage 7. The output signals from the P-layer are proceeded to the recognition Yk-layer and the maximum element is calculated ykj.
n y kj zikj pi ;
i1 i maxk max(Y k ); p u zikmaxid; i i
Stage 9. If r Riterk , then it is assumed that the current active neuron of Y k -layer F2-field, is
If r Riterk , then the current active neuron of Y k -layer frozen and further in the competition is not involved. If all neurons of the current Y k -layer are frozen or completely absent, then proceed to stage 11.
Stage 10. Jump to the next memory level, associated with the current active neuron of Y k -layer by M k relationships.
M k -layer stores semantic relationships of Y k -layer neurons with Y k 1 . If k kc , then proceed to stage 7, otherwise, it is considered that the active neuron of Y k 1 -layer is the one in whose weights the final result of recognition is stored.
Stage 11. Next, a new neuron of Y k -layer is created, m is increased and his weights are trained according to the following rule: z mki d (1 d ) i .
1 d
The proposed structure of the ART-2 network has the following features. 1. The search of an active neuron, in which the result image is stored, now not only based on the frequency of the images of the corresponding classes, but also based on the semantic relationships that are formed at the network learning time. This fact significantly reduces the number of checks, since all memory is now represented in the form of a tree structure. 2. The possibility of parallelizing the calculations in the field F2, which is due to the impossibility of the Y k -layer neuron stimulation, which is not semantically related to the current. u
The experiment was carried out with 10 inputs. As inputs to the analysis, were generated random numbers belonging to the interval (0;1000]. The speed of network training was assessed.
Figure 4 shows the graphs of the dependence of the learning rate on the number of different input images for the classical implementation of the neural network ART-2 and proposed ART-2m, with multi-level memory.
To visualize the results of the network working, used the matplotlib [
10 track vertices 130 track vertices s 10000 track vertices
From the results of the proceeded experiments it follows that when using the proposed memory structure, the network operation time at the selected intervals varies linearly. This is due to the existence of semantic relationships between different levels of the Y-layers in the search of an active neuron. The learning time for a small number of images for ART-2m networks is longer, since more time is required to build a memory tree. But as soon as the number of different filed images exceeds the product of the height of the tree by the average number of elements in the layer, ART-2m shows the best time.
It can be seen from the graphs presented in Figure 4 that as the number of input images increases, the learning time also increases linearly. This allows the proposed modification of the ART-2m network to be used to recognize more different data vectors than when using ART-2.
To eliminate the shortcomings of the classical implementation of ART-2, a modified model of the network of adaptive resonance theory with multilevel memory – ART-2m was proposed. Due to its tree structure of memory, it allows to reduce the search time of a previously stored image. Thus, it is possible to improve recognition accuracy by adding new memory levels with a more tough similarity parameter. The architecture of the ART-2m network allows you to perform image classification much faster, which makes it possible to apply it in real-time systems.
In the future, it is planned to develop an algorithm for parallel ART-2m network and carry out an experiments. It is also planned to apply it in diagnosing of the state of a computer network.