At present, neural network approaches, in particular, the use of ensembles of neural networks, which are an example of a collective solution of problems, have been widely used for modeling complex sociotechnical systems that are difficult to formalize and poorly structured.
The solution quality of the specific task (data mining, forecasting, pattern recognition, classification, etc.) can be significantly improved using neural network ensembles, in which it is expected to form and train a finite set of neural networks, the results of which are taken into account in the overall solution. At the same time, individual decisions are coordinated in such a way that the overall final decision is the best.
One of the fundamental problems of improving the functioning of an ensemble of
neural networks in the subsystems of decision support for information and analytical
systems (DSS IAS) in terms of increasing their accuracy and reliability is the
generation of diversity ensemble (the differences of individual models) [
To resolve this contradiction, it is proposed to use approaches based on the
collective of the neural networks, for the construction of which a new class of artificial
neurons, the so-called selective neurons, differing from classical neurons by a more
efficient method of processing input information approximating the biological neuron
is used [
The models used in the IAS-based IAS based on neural networks have a number of
features and advantages. They are adaptive self-learning systems, which are difficult
to dynamically simulate, and often, it is simply impossible, because They often
contain a significant array of hidden, uncontrolled, incomplete and noisy parameters and
mutual connections between them. Their use makes it possible to solve problems that
are difficult or impossible to solve by traditional methods due to the absence of
formalized mathematical descriptions of the functioning of the object of investigation.
Neural network models have associative memory and in the process of work they
accumulate and generalize information, from which their effectiveness increases with
time. Their use is based on training the neural network to extract information from
experimental data, which ensures the objectivity of the results and increases their
reliability and reliability [
When constructing an ensemble of neural networks, a finite set of previously trained neural networks is simultaneously used, the output signals of which are combined into a joint estimate, which exceeds the quality of the results obtained with the help of local networks entering the ensembles.
The ensemble H ( x̄) of the models hi( x̅) (i=1,2, ... N) is the composition of the algorithmic operators hi: Rd → R and the corrective operation
F: RN → R, in which the final estimate of H( x̄) [
H( x̅) = F(h1( x̄), h2( x̅) ,…, hN( x̅)).
(1)
As is known, the fundamental task in the construction of ensembles is the generation
of a variety of ensemble (or the difference of individual models) [
Obviously, the aggregation of similar models in the ensemble can not lead to a significant improvement in the quality of the solution of the problem.
The problem of generating diversity lies in the fact that individual models of classical neuron networks based on a neuron McCullock-Pitts, are trained to solve one task for one training sample and, as a result, usually quite strongly
Correlated, which affects the accuracy of the solution obtained.
Electoral networks are built on the basis of selective neurons, whose model is close
to the model of a biological neuron. The training of the elective neural network is
carried out not by changing the weight coefficients of the synaptic connections, but by
changing the quantity and quality of the inhibitory and exciting dendrites (signal
transmitters) on the basis of which selective neurons are constructed [
A mathematically artificial neuron is usually represented as a non-linear function of a linear combination of signals with a limited number of inputs. The result of the function is directed to the only output of the neuron, and then to the input of the neuron from the next layer. Scales characterize communication channels, through which signals are received. The task of network training is to select such weights for each connection that would minimize the final value of the output error. However, the classical types of neural networks have a serious drawback – instability to retraining. If the number of weights in the network is large, then the operation of summing up their linear combination will be extremely computationally capacious, and the effect of retraining is likely to arise when the network recognizes images on the training set with extreme accuracy, but it shows a large error percentage on the test set Input data.
Thus, the shortcomings of classical neural networks are a consequence of the use
of weighting coefficients of synaptic connections. Indeed, in a biological neuron,
there is no weight gradation – this is an artificial device. The solution of the main
problems of processing the input signal is achieved by changing the clustering
configuration and the number of dendrites at the input of the biological neuron [
The analysis showed that, by analogy with biological mechanisms, it is advisable
to use a neural network that includes neurons with controlled clustering of the input
channels (synapses) in quantity, and then clustering the channels in quality (exciting
or inhibiting). As an effective classifier of images, it was suggested in [
Cluster formation includes blocking of non-informative communication channels
that do not conduct exciting or inhibitory signals [
K – a cluster of communication channels formed in accordance with the binary code at the input; Σ – adder;
F – a nonlinear threshold function.
Possible characteristic code combinations of the investigated objects at the input of
the selective neuron can be represented in the form of vectors [
x 1 = ( x11 , ..., x1n ) ; … ; x m = ( x m1 , ..., x m n ) , where n is the number of elements of the code combination; M = is the number of objects to be examined. All possible code combinations of input objects form a matrix A, which can be represented as: x11, x12 ..., x1n A = ... ... .. xm1, xm2 ..., xmn Sij = xik xkj = ( xi , x j )
k=1 The sum values S ij are equal to the elements of matrix B:
Let a particular selective neuron contain a cluster of bonds characterized by a code combination x i = ( xi1 , ..., xin ) . When entering the input combination of the input neuron at the input of the neuron, we obtain n
T
B = A* A ,
Where AT is the transposed matrix of A. We obtain in total m х m sums. The largest is the sum:
n Sii = xij x ji = ( xi , xi ) = Ni = N , (6)
j=1 where N i is the number of units in the code combination x i = ( xi1 , ..., xin ) . The property of the sums is that Sij < N , it is used to decide on the recognition of input signals.
Due to the peculiarities of its construction and functioning, the selective neural network provides: • selective recognition achieving of input signals without using the weighting of their synaptic connections; • the possibility achieving of encoding an input signal of a certain type with the channel number or the number of the registering neuron; • input information compression, due to the preservation of only that information about objects that falls into a given channel or registering a neuron; • the speed increase of operation; • the reliability increasing of object recognition when they are large; • achievement of a much greater adequacy of the selective neuron to a biological single-layer perceptron. (2) (3) (4) (5)
Simulation of the time series forecasting process based on an ensemble of selective neural networks has shown that the average error in forecasting individual models of electoral neural networks has decreased by 8-10%. The quality of the diversity of the models of elective neural networks, calculated as the difference between the quadratic error of the ensemble and the average error of individual models of electoral neural networks has improved (increased) by 10-14%. 4
Studies have shown that the use of a new class of neural networks based on selective neurons for the construction of neural network ensembles in subsystems supporting the decision-making of information and analytical systems makes it possible to significantly improve the accuracy and reliability of decision-making by an ensemble of neural networks.
The key difference between the classical McCulloch-Pitts neuron from the selective one is that the model of the latter is close to the model of a biological neuron. The operation of selective neural networks is based on the controlled clustering of the input channels, individual for each neuron, which in turn ensures an increase in the accuracy and reliability of their operation.