The architecture of a multilayer network consisting of several levels of active elements is considered. The input level forms the signals propagated to the connectors (synapses) of the first level. All odd layers of the network consist of connectors (synapses), and even ones consist of switches (neurons). The number of connectors and switches in each layer corresponds to the number of reference signals, the training sample vectors. The process of recurrent adjustment of synaptic connections and neuronal responses of the network is explained both by measures of the similarity of the training sample vectors and similarity measures of these similarity measures. An experimental study using the example of a six-layer network showed that a multi-layer neural-like network of direct propagation is much easier to learn than a recursive network trained by the method of the error back propagation. At the same time, the proposed network is resistant to significant interference when distinguishing signals, which is due to the consideration of additional connections between the components of the reference signals. When analyzing signals against a noise background, under the condition of "interference amplitude / signal amplitude" is less than the average spread of the reference signals, this advantage can become decisive, since it makes it possible to realize an almost error-free signal difference. Multilayered network; direct distribution; vectors of the training sample; measures of similarity of vectors; measures of similarity over similarity measures; discrimination, signal; noise.
Ключевые слова Многослойная сеть; прямое распространение; векторы обучающей выборки; меры сходства векторов; меры сходства над мерами сходства; различение; сигнал; помеха.
The structure of heterogeneous multiply connected network intended for modeling of neurodynamic problems, recognition of signals and images, data processing of phased antenna arrays was proposed in [1]. The implementation of the architecture of such network on clusters of universal and / or graphic processors is briefly described, the approbation of the developed network model on examples of solving a number of known highdimensional neurodynamic problems is performed in [2]. A wide class of neural-like networks was formally described as a structural model of the cybernetic network, where the functional-structural topology of the cybernetic network was also presented for the first time, taking into account the exchange of information and control data streams [3, 4].
In [5] the structure of the multilayered network is proposed, which is a subset of cybernetic networks functioning on the principle of "winner-takes-all" [6]. In this network, both during its training and application, the direct propagation of signals was used. Experimentally, the high efficiency of the network was shown when distinguishing noisy signals. In particular, it was revealed that the level of error of discrimination (the number of incorrect decisions per 100 implementations of interference) of reference signals (with an average variation Var = 13%) on which the network was trained, nonlinearly depends on values of the ratio N0 = "amplitude of noise / amplitude of signal". At the same time, the error is practically zero at N0 < Var = 14% and reaches 10% at N0 = 18%.
The parallel software architecture was described, based on the object-oriented approach and on the known GoF design patterns [7, 8]. The template "factory method" is used to expand the class of network simulators, and the "Bridge" template allows building various implementations for CPU and GPU platforms. The architecture is intended for simulation of multidimensional problems (network neurodynamics, compression of multidimensional data, pattern recognition) and, in particular, multi-layer networks from [5].
In this paper, architectural solutions for learning a multi-layer network are discussed in more detail.
The aim of the work is to consider the principle of learning a multilayered network – adjusting its synaptic connections and neural responses according to similarity measures of vectors of the training sample and similarity measures over these similarity measures. This consideration also explains the high efficiency of the discrimination of noisy signals.
An example of the structure of the multilayered network of the direct propagation is shown in Fig. 1 [5, 8].
As seen in Figure 1, the network consists of several layers of active elements. The input layer forms the input signals propagated to connectors (synapses) of the first layer. All odd layers consist of connectors (synapses), and even ones consist of switches (neurons). The number of synapses and neurons in each layer corresponds to the number of reference signals – vectors of the training sample. Figure 1 shows an example of an already trained network, with the memory register of each connector of the first layer containing the corresponding reference vector of the training sample. The first reference signal without interference is fed to the input layer. Therefore, every even layer of the network is unmistakably, with a measure of similarity 1, identifying the input signal, as Signal 1.
Principle of the learning of neurosimilar network is that each synapse’s memory register of an starting with the third one, records the responses of all neuron-switches of previous layers to the reference signals sequentially fed to the network input in the learning process. odd la
The response of each neuron of any layer to the input signal – the vector Z at the input of the previous synaptic connector – is formed as an odd power of cosine of the angle between the input vector Z and the vector X stored in the connector register: ( , )= cos2 +1( , ). In the above example, n = 20, which provides a strong nonlinearity – the resonance response of the neuron-commutator to the input signal.
Let us consider in more detail the iterative process of learning the network. To simplify the drawings, the controllers controlling the configuration and operation of connectors are not shown on them.
At the first training step of the network, each reference training sample signal is written to the memory register of the corresponding synapse-connector of the first network layer, as shown in Figure 2.
In the second step of network training (Figure 3), all reference signals of the training sample are sequentially fed to the input layer of the network, and for each reference signal (in Figure 3 – signal 1), the responses of all switch-neurons of the second layer are written to connector’s registers of the third layer. As a registers of connectors of the third layer, similarity measures ( , )= cos2 +1( , )of all vectors of the reference training sample are written (Figure 4).
In the third step of network training (Figure 5), all reference signals of the training sample are sequentially fed to the input layer of the network, and for each reference signal (in Figure 5 – signal 3), the responses of all switchneurons of the fourth layer are written to connector’s registers of the fifth layer. As a result, connectors of the fifth layer, similarity measures under similarity measures (( , ), ( , ))= cos2 +1(( , ), ( , ))of all vectors of the reference training sample are recorded (Figure 5). result, in
As a result of the first experiment, we present in Figure 6 an illustrative example of the network distinguishing the third reference signal against a noise background when N0 = "amplitude of noise / amplitude of signal" = 0.25, that corresponds to the ratio Noise/Signal = 6.25%. gives an incorrect recognition of the input noisy signal.
However, in the subsequent odd layers, where similarity measures are compared with the reference similarity measures, responses are formed that correctly identify the input noisy signal.
For an experiment as a test example were chosen 6 components of 6 reference signals Sk = (sk1, .., sk6)T (k = 1, 2,.., 6), provided in the table 1.
Apparently from the table 1, the average variation (Var) of signals makes 13%. At the same time: = 16 ∑6=1 = 16 ∑6=1 [∑6 =1(
− 2]1/2 [∑6 =1 2]1/2 , 〈 〉 = 16 ∑6=1 .
The task consists in a research of dependence of an error of the distinction of reference signals from table 1 from the amplitude of an additive hindrance N. (1) (2) (3) where random( )– is the size which is evenly distributed in the range of (-1, 1), and N0 – the maximum relation of amplitude of a hindrance to amplitude of a useful signal.
For descriptive reasons reference signals from table 1 are given in the figure 7, and one of signals (S3) with the maximum variation of amplitudes in 18% in the presence of a hindrance with N0 = 13% is given in the figure 8. In this case the observed signal X is expressed as:
In an imitating model experiment the hindrance was generated as a vector N = (n1, n2, nm)T of the random evenly distributed sizes:
= + . = 0 ∗ ∗ (1 − 2 ∗ random( )),
Projections of all reference signals to the plane (X, Y) carried out under technology [9] are given in the figure 9 at one of realization of a hindrance with N0 = 13%. From the figure 9 it is especially visually visible that in the presence of a hindrance it is probably wrong to identify an observed signal Z1 with the reference signal S3, and Z2 with S4.
From statistics of errors of charts (figure 10) it is visible that in comparison with the 2nd layer the 6th layer of network gives a prize in reliability of recognition from 1% to 4%.
The carried out research showed that the multilayered neurosimilar network of direct distribution gives some advantage at distinction of noisy reference signals. This advantage is caused by accounting of additional communications between components of reference signals. In the analysis of signals against the background of hindrances with N0 = "amplitude of noise/amplitude of signal" of average variation of reference signals this advantage can become decisive as allows us to make almost faultless distinction of signals.
The interesting result turns out when giving on an entrance of the trained network of any signal, considerably different from all reference signals.
So, in the figure 11 the example of responses of network to an entrance signal XT = (11,07; 17,92; 3,14; 23,51; 12,78; 14,90) is shown. At the same time, the network shows on the similarity of an entrance signal with the 3rd reference signal 3= (16,00; 14,00; 3,00; 21,00; 6,00; 16,00), though degree of this similarity is extremely small (0,25). At the same time, the 4th and 6th layers carry an entrance signal to the 3rd reference signal.
In the figure 12 the example of responses of network to an entrance signal XT = (13,47; 14,91; 0,02; 29,01; 12,05; 5,51) is shown.
In this example the 2nd layer of network shows on the similarity of an entrance signal with 4rd reference signal 4 = (20,00; 9,00; 7,00; 22,00; 8,00; 11,00), and the 4th and 6th layers carry an entrance signal to the 3rd reference signal.
Thus, the considered multilayered network of direct distribution of signals with the adjustment on measures of similarity of vectors of the training selection is very simple for the process of training and has the increased reliability of recognition of reference signals in the presence of hindrances in comparison with one layer.
The work was supported by the state in the person of the Ministry of Education and Science of Russia by lot code 2017-14-579-0002 on the topic: "Development of effective algorithms for detecting network attacks based on identifying of deviations in the traffic of extremely large volumes arriving at the border routers of the data network and creating a sample of software complex for detection and prevention of information security threats aimed at denial of service". Agreement No. 14.578.21.0261 on granting a subsidy of September 26.2017, a unique identifier of the work (draft) RFMEFI57817X0219.
References dlya modeli «Informika Об авторах: Краснов Андрей Евгеньевич, доктор физик-оматематических наук, профессогрл,авный научный сотрудник, оГсударственный научн-оисследовательский институт информационных технологий и телекоммуникациa.йk,rasnov@informika.ru Надеждин Евгений Николаевич, доктор технических наук, професгслоарв,ный научный сотруд н,ик Государственный научн-оисследовательский институт информационных технологий и телекоммуникаций, e.nadezhdin@informika.ru Никольский Дмитрий Николаевич, кандидат физик-оматематических наук, доценветд,ущий научный сотрудник, Государственный научн-оисследовательский институт информационных технологий и телекоммуникациdй.n,ikolsky@informika.ru Шмакова Елена Германовна, кандидат технических наук, доцент, заведующая кафедрой «Информационных систем, сетей и безопасн,оРсотсиси»йский государственный социальный университет, rusja_lena@mail.ru