The theoretical and mathematical substantiation of standard and selective neural network technologies is given. Models have been developed for visual modeling of processes in standard neural networks based on McCulloch-Pitts neurons and selective based on selective neurons. An assessment of the accuracy of recognition in selective neural networks is given, based on an assessment of the fulfillment of the basic conditions for recognition. The neuroeducational system allows effective teaching of neurotechnologies to schoolchildren, students, and specialists in related professions.
The theoretical and mathematical substantiation of standard and selective neural network technologies is given. The neuro-educational system for teaching neural network technologies and their applications based on standard and selective neural networks is designed for pupils, students, graduate students, people of the "silver age", specialists in related professions. The possibility of solving more complex problems using the developed software, accessible to a wide audience of schoolchildren, students, specialists in related professions, has been realized. This is the calculation of the weight coefficients of single-layer and multi-layer neural networks with McCallock-Pitts neurons using the selective Monte-Carlo method. Familiarization with the methods of "deep learning" in neural networks.
The neuro-educational system is made in two versions. The first option is a material instrumental implementation in the form of training electronic working models.
The second option is computer implementation in the form of separate programs for the learning process and the use of more complex training tasks.
Material instrumental implementation in the form of training electronic operating mock-ups intended for training standard neurotechnologies based on McCallock-Pitts neurons and selective neurotechnologies based on selective
neurons. For training standard neural networks, a visual technology based on the Monte-Carlo selective method has been developed, the programs are presented in the form of executable exe files.
In the second computer version, more complex training examples for recognizing objects on a large-screen monitor are considered. The developed software allows you to go to the solution of practically useful tasks of object recognition and control tasks using selective neurotechnologies. Tasks are developed that illustrate the training methods used in neural networks of "deep" learning.
In order for the neural network to be able to complete the task, it needs to be
trained. Training the most common neural networks using McCallock-Pitts
neurons comes down to calculating its weighting coefficients [
For effective training in neuro-educational technologies, a series of software tools have been developed, as well as the material implementation of the neuroeducational system.
For educational purposes, a single-layer perceptron neuro-educational system based on three McCallock-Pitts neurons was developed. The perceptron was supposed to recognize the letters L, T, X at the weight coefficients generated during the training. The proposed perceptron was implemented in the form of an electrical circuit shown in Fig. 2.
From the many shortcomings of neural networks based on training by
selecting weights, selective neural networks using selective neurons described in
[
Fig. 4. The structure of a single-layer Rosenbluth perceptron using McCallock-Pitts neurons on the left and a selective perceptron on the right
In fig. 4: K - formed clusters of communication channels; - adder, F
threshold nonlinear elements. Triangles indicate blocked communication
channels from among the input ones that are not essential for objects at the input
of the perceptron. The effectiveness of the selective perceptron for the
recognition of contour objects follows from the theorem given in [
The structure of a material device was developed that implements the instrumental method of training a selective neural network using the example of 3 selective neurons and 3 input signals characterized by binary sequences of 9 cells, shown in Fig. 5
In fig. 5 for clarity, the recognition of 3 vector objects is modeled. Selective clustering is illustrated by the first three vertical blocks in the structural diagram. Blocking channels and the formation of clusters is done using black triangles. Summation of the elements of vectors xi (xi1,..., xi9 ) (i 1, 2, 3) is performed in the summation blocks. Then a nonlinear threshold transformation is performed in the blocks indicated F .
To justify the uniqueness of recognition of contour images by a selective neural network, a pixel scan of normalized contour objects was implemented, matching each object with a binary string of 0 and 1 (0 - no contour, 1 - contour point). Then for mismatching contour objects the relation holds ( xi , x j ) N
(i 1,..., l) . We believe that it is impossible to combine
contour objects with a linear transformation O1, ..., Ol , which allows us to realize
the linear independence of the binary codes of these objects. A uniqueness
recognition theorem for incompatible contour objects has been proved [
Theorem: Suppose that in a two-dimensional region divided into pixels by a rectangular lattice, m contour objects are given that are incompatible with movements — shift, rectangular transfer, and rotation. Let the objects be scanned using a horizontal scan into binary sequences - vectors of 0 and 1 in length n , that is i = (1,..., m) . Let all possible code xi = (xi1,..., xin ) combinations of input signals be collected in matrix А x11, x12..., x1n x , x ..., x A = 21 22 2n
... ... ...
xm1, xm2 ..., xmn
Let a particular neuron contain a cluster of connections characterized by a code combination in the form xi = (xi1,..., xin ) . Consider the amount ij
n Sij xik xkj ( xi , x j ) .
k 1 in the form АхT , where хТ j j We represent the sums S is the transposed column vector from the row vector of the vector хk . We represent all possible sums Sij (i 1,..., m; j 1,..., m) , the number of which is equal m2 , in the form of the matrix B where AT is the transposed matrix A . Consider the amount S ii
B AAT , n Sii j1xij x ji ( xi , xi ) Ni ,
Sij Sii N .
where Ni is the number of units in the code combination xi = (xi1,..., xin ) . We use the property to recognize input objects
Then the recognition of each of m input objects by the considered singlelayer perceptron will be unique.
The theorem states that the solution to the problem of the selective perceptron can be found in one iteration, that is, for a time interval much shorter than during the iterative process for the Rosenblut perceptron. In this case, the solution itself will be the only one.
The selective perceptron based on selective neurons has several advantages over the Rosenbluth perceptron based on McCallock-Pitts neurons. We list the useful properties of a selective perceptron that can be achieved: 1. Solving recognition, control, and other problems when using the instrumental method in selective neural networks is unambiguous under certain conditions, while recognition in neural networks with McCallock-Pitts neurons is ambiguous; 2. The training procedure for selective neural networks is simplified, it becomes available to a layman; 3. Reduces training time; 4. Training can be achieved by tools without the use of computing tools.
The selective perceptron on selective neurons has been implemented in practice. The electrical circuit of the selective neuron provides a light indication of all involved communication channels of the neuron with the control panel, which allows you to visualize the clustering of working communication channels. The control panel is common to all three selective neurons that make up the selective perceptron. The selective adjustment of the communication channels of a selective neuron to informative cells of input signals is performed with the corresponding internal communication channels located in the selective neuron itself. In this case, the active communication channels included in the selective cluster are identified by ignition of the LEDs located on the neuron. Behind the indicators of communication channels is a neuron excitation recorder.
The elimination of the shortcomings of iterative algorithms can be achieved using an innovative calculation procedure based on the selective Monte-Carlo method. Inequalities for determining weights have innumerable solutions. Various iterative procedures are used to find these solutions. However, what solution we will find after the iteration procedure is unknown. Some clarification of the situation is given by the well-known theorem of the American mathematician Novikov. Its essence is that it claims, under a number of restrictions (rather stringent), that learning - the iterative process will end in a finite time. And what set of weights and what quality of the system we get is unknown. The developed program allows you to find training weights directly using the selective Monte-Carlo method. The method allows you to find a large number of sets of weighting coefficients and selects those that satisfy the given optimality conditions. You can find sets lying in a given numerical range, sets that provide a stable mode of operation of a neural network, and sets that exclude ambiguous recognition.
The software is made in the programming language Matlab7. We present the results of testing the developed program. Consider an example of calculating weights for one of the training options. The calculation results of one possible set of weights for the signals, each of which is a binary sequence of 9 cells from 0 and 1, are presented below. The matrix of weights for binary input signals is equal to 0.9113 0.1775 0.3599 1.4603 0.4626 0.2949 1.2842 1.4467 0.3703 0.9878 1.0770 0.7659 0.2474 0.8311 0.3239 0.5889 1.1790 0.3721 0.4015 0.1678 1.1979 0.7318 1.3805 0.8073 0.8069 0.3543 1.0721 The initial matrix of input signals is 1 0 0 1 0 0 1 1 1 1 1 1 0 1 0 0 1 0 1 0 1 0 1 0 1 0 1 The matrix that controls the quality of the found set of weights is 5.4729 3.3580 3.3884 3.3752 4.8407 3.5458 3.3667 3.5020 4.8589
The quality of the found weighting coefficients can be judged by the homogeneity of the diagonal members of the matrix; in accordance with the calculations, the diagonal terms must be in the interval 4.8 Cii 5.9 . In addition, the condition of "high quality recognition" must be met Cij Cii .
This condition is also fulfilled. The number of calculation cycles is 375533. The counting time is less than 1 second.
A certificate of state registration of computer programs has been received for the software “The program for calculating perceptron weight coefficients using the selective Monte-Carlo method”. 5
On the control panel that simulates the receptors of the input signals, sequentially activate the receptors corresponding to the letters L, T, X. The recognition indicators for the letters L, T, X should light up. Identification indicators of unidentified neurons should not light up. The entire learning process is illustrated by the Rosenbluth perceptron, which includes three McCallock-Pitts neurons. The single-layer Rosenblut perceptron in the recognition mode of the letters of the English alphabet L, T, X is shown in Fig.6. Consider the recognition of the letters of the English alphabet L, T, X using a selective perceptron.
The appearance of a selective perceptron based on 3 selective neurons, designed to recognize three input objects in the form of letters, numbers and other characters that can be set on the control panel screen, is shown in the photograph in Fig. 7.
In fig. 7, the perceptron recognizes the letter L from the top left, the perceptron recognizes the letter T from the top right, the perceptron recognizes the letter X from the bottom center.
Consider the binary coding of 10 digits 0, 1, ..., 9 on a 4x6 monitor screen. The numbers on the monitor screen and their decomposition into binary sequences are shown in Fig. 8.
Fig. 8. The location of the numbers on the monitor screen and their decomposition into binary sequences
Decomposition of 10 digits from 0-9 in the form of a binary string. The significance of each expansion in the form of normalization according to the norm is taken into account. Each digit corresponds to a binary string of 24 cells of 0 and 1. The totality of all lines is shown in Fig. on the right, in the form of a 24x10 matrix. The binary input signals of the objects are shown in rows of a 24x10 matrix.
For training, other more complex recognition options were implemented
using an instrumental method based on the use of selective neural networks.
This is a "smart" recognition of electrocardiograms. The well-known method of
ECG recognition is based on the determination of individual characteristics of
an ECG type: maximums, minimums, inflection points, etc. Smart ECG
recognition based on selective neural networks is based on a comparison of a
real ECG with a reference from a database. Software has been developed that
allows you to enter a real ECG and recognize its belonging to one of the ECGs
from the database [
The estimation of the recognition accuracy in neural networks is estimated using the relation M M 0 100% ,
M where M - the total number of recognitions, M0 is the number of erroneous recognitions in practice. For selective neural networks, it is expedient to evaluate the recognition accuracy by the accuracy of the fulfillment of the basic recognition conditions
Sij U p Sii U p , where Sij ( i, j 1,..., m ) – integral sums of input signals xi (xi1,..., xin ) , n - number of vector components xi . Integral sums
the matrix B signals
xi .
U p min Sii .
i где AT - matrix transposed to A , A - matrix composed of vectors of input The threshold value
U p is determined from the ratio Let us define a function
number of condition failures Sij U p determined from the relation . Then the recognition accuracy is B A AT , m2 M 0 100% , m2 where m2 number of matrix elements B .
You can give a visual geometric interpretation of the number of misidentifications. To do this, you need to build a graph of the function Z B(i, j) in three-dimensional space (i, j, Z ) . Incorrect recognition is characterized by off-diagonal matrix elements that will be larger than diagonal elements.
Let's give an example of evaluating the accuracy of a function Z B(i, j) when recognizing 10 digits (0, ..., 9) in a 4x6 field. Each digit corresponds to a binary string of 24 cells from 0 and 1. The totality of all strings is shown in Fig. on the right, in the form of a 10x24 matrix. Binary input signals characterizing objects are shown by rows of a 10x24 matrix. A geometric illustration of the selectivity of a single-layer perceptron based on selective neurons is shown in Fig. 9.
Threshold value U p = 12. As you can see from the graph of the function B(i, j) , all values Sij U p 12 . Therefore, M 0 = 0 and recognition
The theoretical and mathematical substantiation of standard and selective neural network technologies is given. Mock-ups have been developed for the visual simulation of processes in standard neural networks based on McCallockPitts neurons and selective ones based on selective neurons. The neuroeducational system allows for effective training in the neurotechnology of senior schoolchildren, students, and specialists in related professions. Examples for teaching more complex recognition without mathematics in computer implementation are considered.
The article was prepared with the support of the Russian Foundation for Basic Research, grants No. 18-07-00918, 19-07-01137 and 20-07-00926.