The diagnosis stage plays a crucial role in medicine. Timely accurate diagnosis facilitates the choice of therapy and significantly increases the probability of treatment of the patient. The using of neural networks is one of the ways to improve the efficiency of medical diagnosis [1].
The accuracy of the diagnosis and the speed with which it can be delivered depend on many factors: the patient's condition, the available data on the symptoms and signs of the disease and the results of laboratory tests, the total amount of medical information on the observation of such symptoms in a variety of diseases and, finally, the qualification of the doctor. A major role in this process is played by the human factor, which often leads to errors [1], [2]. Some of the specific medical diagnosis difficulties that need to be considered are listed below [3].
The basis for a reliable diagnosis is a wealth of practical experience, which can be reached only the middle of a doctor's career and is absent at the end of training, of course. This is especially true for rare or new diseases, where experienced doctors are in the same situation as beginners.
The quality of diagnosis depends on the skill, knowledge and intuition of the doctor.
Emotional problems and fatigue adversely affect the work of the doctor.
Training of specialists is a long and expensive procedure, and therefore in many, even in developed countries, there is a lack of skills.
Medicine is one of the fastest growing and developing fields of science. New results disqualify the old ones, new drugs appear every day. The same applies to the diseases themselves, which take new forms.
These factors necessitate the search for new solutions and tools, for example, the use of artificial neural networks (ANNs) [2].
The ANN technologies are designed to solve difficult-to-formalize problems, which, in particular, are reduced to many problems of medicine [4], [5]. This is primarily due to the fact that the researcher is often provided with a large number of heterogeneous factual material for which a mathematical model has not yet been created. In addition, it is necessary to present the results of the analysis in a form, which will be understandable to the specialist. So ANN is a powerful and flexible method of simulating processes and phenomena. Neural networks can be different in structure and form, but they have several common features. A distinctive feature of neural networks is their ability to training on the basis of experimental data of the subject area. With regard to medical subjects, experimental data are presented in the form of a set of initial features or parameters of the object and the diagnosis based on them. Training of ANN is an interactive process in which the neural network finds hidden nonlinear relationships between the initial parameters and the final diagnosis, as well as the optimal combination of weight coefficients of neurons connecting adjacent layers, in which the classification error tends to a minimum [6]. In the training process, the input of the neural network is fed a sequence of initial parameters along with the diagnoses that characterize these parameters. Careful formation of the training sample determines the quality of work, as well as the level of error of the neural network.
A number of difficulties are associated with the use of neural networks in practical problems. One of the main problems of application ANN technologies is a previously unknown degree of complexity of the projected ANN, which will be enough for a reliable diagnosis. This complexity can be unacceptably high and will require more complex network architecture. It is known, for example, that the simplest single-layer neural networks are able to solve only linearly separated problems [7]. This limitation can be overcome by using multilayer neural networks.
The basis of ANNs are neurons with a structure similar to biological analogues. Each neuron can be represented as a microprocessor with several inputs and one output. When neurons are joined together, a structure is formed, which calls a neural network. Vertically aligned neurons form layers: input, hidden and output. The num-ber of layers determines the complexity and, at the same time, the functionality of the network, which is not fully investigated.
For researchers, the first stage of creating a network is the most difficult task. The following recommendations are given in the literature [8][9][10].
1. The number of neurons in the hidden layer is determined empirically, but in most cases the rule is used
, where h N is the number of neurons in the hidden layer, i N in the input and o N output layers.
2. Increasing the number of inputs and outputs of the network leads to the need to increase the number of neurons in the hidden layer. 3. For the ANNs modeling multistage processes required additional hidden layer, but, on the other hand, the addition of hidden layers may lead to overwriting and the wrong decision at the output of the network.
Based on these recommendations, the number of layers and the number of neurons in the hidden layers is chosen by the researcher, based on his personal experience.
The ANN are attractive from an intuitive point of view, because they are based on a primitive biological model of nervous systems. In this connection, there is an assumption that to improve it may be appropriate to apply another borrowing from nature, for example, evolutionary calculations and, in particular, neuroevolution. Neuroevolution in this paper refers to the automatic modification of neural networks using genetic algorithms. With this methodology, possible variations of neural networks with different topologies are grown, which with each iteration, called generation solve the problem better. Despite genetic programming, as well as evolutionary calculations in general, do not guarantee finding the optimal result, this approach eventually allows us to come to the results applicable to solving practical problems. However, it will take a reasonable amount of time to achieve such results. Thus, the level of complexity of the neural network that arises when it is necessary to create a neural network is significantly reduced, because when it is created, it is only necessary to select the parameter that evaluates the work of the neural network and provide a suitable set of data.
Despite the fact that most of the works devoted to the neuroevolutionary approach offer only a theoretical approach to solving problems of neural network optimization, it is possible to find several promising and noteworthy methods [11][12][13][14].
From the early works of noteworthy cellular Frederick Gruau method [15], [16] uses a special grammar for the representation of neural network structures. One individual represented an entire neural network, with each neuron considered as a biological cell, and the growth of the network was determined through the mechanisms of sequential and parallel "division" of neurons -cells. However, this method involves the implementation of a large number of specific operators that provide simulation of cell activity.
The Hierarchical SANE (Symbiotic, Adaptive NeuroEvolution) [17] method uses a different approach. It is consider the development of two independent populations, one of which individuals are separate neurons, and the other contains information about the structures of an artificial neural network. The disadvantages of this method include the fact that the number of hidden neurons and connections is limited.
The ESP method [18] is a development of the sane method. Its main difference is that the network structure is fixed and is given a priori. The population of neurons is divided into subpopulations, in each of which the evolution is independent. Due to parallelization of the solution search, as well as simplification of the problem due to the rejection of the evolution of the artificial neural network structure, ESP works much faster than SANE, sometimes by an order of magnitude, but for the successful operation of the method it is required to choose the appropriate structure of the neural network [19].
One of the most potentially successful attempts to get rid of the disadvantages of direct coding while preserving all its advantages is the method proposed in 2002, called NEAT -Neural Evolution through Augmenting Topologies [15], [20]. Designed by Kenneth Stanley, the NEAT method allows to customize the structure of the network, and without restrictions on its complexity. The solution proposed by the authors is based on the biological concept of homologous genes (alleles), as well as on the existence in nature of the synapsis process -the alignment of homologous genes before the crossover. The technique assumes that two genes (in two different individuals) are homologous if they are the result of the same mutation in the past. In other words, with each structural mutation (gene addition), a new gene is assigned a unique number, which then does not change during evolution. The method uses a number of techniques, such as historical labels and specialization of individuals, to make the process of evolution significantly more efficient [21].
Summing up, it can be noted that the joint use of evolutionary methods and artificial neural networks allows us to solve the problems of configuration and training of artificial neural networks both individually and simultaneously. One of the advantages of this synthesized approach is largely a unified approach to solving a variety of problems of classification, approximation, control and modeling. The use of qualitative evaluation of the functioning of artificial neural networks allows the use of neuroevolutionary methods to solve the problems of the study of adaptive behavior of intelligent agents, the search for game strategies, signal processing. Despite the fact that the number of problems and open questions concerning the development and application of neuroevolutionary methods (coding methods, genetic operators, methods of analysis, etc.) is large, often for the successful solution of the problem with the use of neuroevolutionary method adequate understanding of the problem and neuroevolutionary approach, as evidenced by a large number of interesting and successful works in this direction [15].
In the method, which is proposed to find a solution using a population of neural networks: . This allows for a uniform distribution of single and zero bits in the population to minimize the probability of early convergence of the method ( min p ). After initialization, all individuals have coded networks in their genes with-out hidden neurons (N h ), and all input neurons (N i ) are connected to each output neuron (N o ). That is, at first, all the presented ANNs differ only in the weights of the interneuron connection wi. In the process of evaluation, based on the genetic information of the individual under consideration, a neural network is first built, and then its performance is checked, which determines the fitness function ( fitness f ) of the individual. After evaluation, all individuals are sorted in order of reduced fitness, and a more successful half of the sorted population is allowed to cross, with the best individual immediately moving to the next generation. In the process of reproduction, each individual is crossed with a randomly selected individual from among those selected for crossing. The resulting two descend-ants are added to the new generation
Once a new generation is formed the mutation operator starts working. However, it is important to note that the selection of the truncation significantly reduces the diversity within the population, leading to an early convergence of the algorithm, so the probability of mutation is chosen to be rather large ( 25% -15 mut p
) [22]. If the best individual in the population does not change for a certain number of generations (by default, it is proposed to set this number at eight), this individual is forcibly removed, and a new best individual is randomly selected from the queue. This makes it possible to realize the exit from the areas of local minima due to the relief of the objective function, as well as a large degree of convergence of individuals in one generation. The general scheme of the method demonstrated at Fig. 1.
It is obvious that the chosen method requires special genetic operators that implement crossover and mutation. At crossover two parental individuals which produce two descendants are used. Common neurons and connections are inherited by both offspring, and the value of connections in the networks of descendants are formed by a two-point crossover. Elements of ANN, of distinct played out between generations.
An important feature is that neurons with the same indices are considered identical, despite the different number of connections and position in the network, as well as the fact that one of these neurons could have a different index, which changed as a result of correction of indices after mutation. For this purpose, three coefficients were introduced that regulate the size and direction of the network.
The first of them characterizes the degree of connectedness of neurons in the network and is calculated by the formula:
where c N is the number of connections in the network, i N , o N , s N are respectively, the number of input, output neurons and the total number of neurons in the network, FB is a variable indicating the permitted occurrence of feedbacks ( FB =1) or not ( FB =0). It is worth noting that connections from hidden neurons to the output can appear in any case. Thus, the smaller con f the more likely a new relationship will be added as a result of the mutation [23]. The use of the second coefficient is based on the assumption that the more elements in the sum of the input and output vectors of the training choice (the greater the total number of input and output neurons), which is probably a more complex network is necessary to solve the problem. The second coefficient is calculated by the formula:
That is, the more neurons in the network, the less will be diff top f . and the less likely will be selected mutation that adds a new hidden neuron [23].
The third criterion is also based on the assumption that a more complex network should be used to solve more complex problems. However, this criterion characterizes the conditional complexity of the network. This criterion is based on the concept of cyclomatic complexity [24], [25].
( 3 ) For any of the described cases, the algorithm uses a ligament
, because for use it is necessary to take into account the degree of connectivity of already existing neurons. Thus, using mutations can be pointwise to change the parameters of the structure of the ins.
Chaotic the addition (removal) of neurons and connections can lead to situations where, for example, in a network of many neurons and few connections. It would be more logical to apply different types of mutations depending on the features of the network architecture represented by the mutating individual [26][27][28].
Removing links gives a side effect: there may be hanging neurons that have no incoming connections, as well as dead-end neurons, that is, without output connections [26], [27], [29]. In cases where the function of neuronal activation is such that at zero weighted sum of inputs its value is not equal to zero, the presence of hanging neurons makes it possible to adjust the neural displacement. It is worth noting that, on the other hand, the removal of links may contribute to the removal of some uninformative and uninformative input features.
Consider the dependence of the type of mutation on the values con f , . This approach, on the one hand, does not limit the number of hidden neurons from above, on the other hand, it prevents the immeasurable increase of the network, because the addition of each new neuron to the network will be less likely. The mutation of the weight of a random existing bond occurs for all mutating individuals with a probability of 0.5.
is necessary in order to change the number of neurons adequately network topology, because the addition (removal) of neurons need information about the feasibility of changes. This information can be obtained indirectly from the value of the characteristic.
On condition using the support vector machine, the optimality criterion for calculating the output weights may not be specified. If the value of the mean square error is replaced by the criterion of the maximum separation of the support vectors, then the optimal linear weights of the output can be estimated using, for example, quadratic programming, as in the traditional method of support vectors, for this it is advisable to use the Evoke operator [30], by the formula:
where
w , is weights corresponding to k training sequences i , each length i l , and are calculated using the support vector machine. The value of the mean square error is replaced by the criterion of maximum separation of support vectors. In this case, the optimal linear weights can be estimated using quadratic programming, as in the traditional support vector machine.
One of the problems of neuroevolutionary method realization is the algorithm of ANN output calculation with arbitrary topology.
ANN can be represented as a directed planar graph. Based on the fact that the network structure can be any, loops and cycles containing any nodes are allowed in the graph, except for the nodes of the corresponding input neurons. Let denote the set of nodes of the graph by
, and a set of arcs through
, where v N and e N are accordingly, the number of nodes and arcs in the graph, and
, the weight of the corresponding link will be denoted by l k w , . Give the index to the nodes of the graph as neurons, that is, the nodes that are the input neurons, called input, have an index out of range
. By analogy, the indexes of outgoing nodes belong to the interval
, and indexes for hidden nodes will be set in the interval
. Let introduce an additional characteristic for all nodes of the graph equal to the minimum length of the chain to any of the input nodes and denote it i ch . Let's call i ch the layer to which the i th node belongs. Thus, all input nodes belong to the 0 th layer, not all input nodes that have input arcs from the input belong to the 1 st layer, all other nodes with input arcs from nodes of the 1 st layer will belong to the layer with index 2, etc .in this case, there may be situations when the node does not have input arcs, we will call it a hanging node with the layer number 1 i ch .
For arcs, we also introduce an additional characteristic l k b , for the arc l k c , , which is necessary to determine whether the arc corresponds to forward or reverse. It will be calculated as follows:
That is, if the index of the layer of the end node of the arc is greater than the index of the layer of the beginning node, then we will consider such an arc as a straight line, otherwise we will consider the arc as an inverse.
Since each node of the graph represents a neuron, we denote by i sum the value of the weighted sum of inputs, and through i o is the value of the output (the value of the activation function of the i th neuron-node). Then,
where fitness f is the function of neuron activation.
Let's divide the whole process of signal propagation from the input nodes into stages, and during one such stage the signals manage to pass only one arc. The number of the stage is denoted by s. For the very first stage s=1. For short assumed that all arcs have the same length, and the signals are sewn on them instantly. We denote the feature that the output of node i was updated at this stage through i a , that is, if 1 i a , then the output of the node at stage s is calculated, otherwise, if 1 i a
-not. Let's introduce one more designation
it vector of input signals. Then the algorithm for calculating the ANN output is as follows:
, for all
, for all
, than go to the step number 3;
2. for all input arcs
, than
The stopping criterion of the ANN output calculation algorithm can be one of the following:
─ stabilization of values at the output of ANN; ─ s exceeds the set value.
It is more reliable to calculate the output until the values at the output of ANN do not change, but for the case when the network contains cycles and/or loops, its output may never become stable. Therefore, the required additional stopping criteria limiting the maximum number of stages of calculation of network output. For networks with no feedback ( FB =0) in many cases, allow the
During testing, the main task is to track the speed of the proposed method, quality and stability. Since synthesized ANN can be further used as diagnostic models for medical diagnosis, testing should be carried out on the relevant test data. Also, testing will be carried out in 2 stages: the first stage will consist in the synthesis of ANN only with the help of modified genetic algorithm, and the second -in additional processing of the initial layer by the support vector machine. This strategy will allow to know more clearly how useful the support vector machine is. Data for testing were taken from the open repository -UC Irvine Machine Learning Repository. Data sample was used: Breast Cancer Coimbra Data Set [31]. Clinical features were observed or measured for 64 patients with breast cancer and 52 healthy controls. There are 10 predictors, all quantitative, and a binary dependent variable, indicating the presence or absence of breast cancer. The predictors are anthropometric data and parameters which can be gathered in routine blood analysis. Prediction models based on these predictors, if accurate, can potentially be used as a biomarker of breast cancer. Table 1 shows the main characteristics of the data sample. During the evaluation of the test results we will pay attention to the following criteria:
─ the spent time, s; ─ average error of final network ( E ); ─ standard deviation (SD).
The relative error value in this case will be calculated as the ratio of the classification error to the total sample size (number of instances).
where E is relative error; Standard deviation gives an idea about how one or the other, the ANN accurately predicts the user's rating, since the estimation is calculated the difference between the result of the work of ANN's and known result. It is also important to know that this indicator can be calculated only with a sufficient amount of observations. Otherwise, the calculation of the SD will be uninformative and its use will not lead to improvement of the results of the ANN.
where SD is standard deviation, i x is i th element of the set, sampl Number the number of instances in the sample, x is the mean value of these observations [32].
Table 2 shows the results of testing the modified genetic algorithm in comparison with the NEAT and ESP methods. As the table shows, the modified GA according to the time of fulfillment is ahead of ESP in time, however, inferior to the NEAT. However, it should be noted that in the analysis of error values, the proposed method is significantly ahead of existing methods.
Let's repeat testing, but now with additional use of the support vector machine. The results are shown in Table 3. As can be seen from the table, the modified GA using the support vector machine is inferior to the opponents in terms of execution time. However, on indicators of minimum, maximum errors and average errors of the output ANN significantly better than their competitors. Therefore, we can conclude that the use of the support vector machine really significantly improves the results of the synthesis. As you can see from the diagram, the modified genetic method was more iterative than the existing methods, but the time spent on iteration was less. That is, it can be concluded that iterations are not complex and for their reduction it is possible to resort to parallelization, which will significantly speed up the work even when using the support vector machine [33][34][35].
The problem of finding the optimal method of synthesis of ANN requires a comprehensive approach. Existing methods of ANNs training are well tested, but they have a number of nuances and disadvantages. The paper proposes a mechanism for the use a modified genetic algorithm for its subsequent application in the synthesis of ANNs.
Based on the analysis of the experimental results, it can be argued about the good work of the proposed method. However, to reduce iterativity and improve accuracy, it should be continued to work towards parallelization of calculations.
The work was performed as part of the project "Methods and means of decisionmaking for data processing in intellectual recognition systems" (number of state registration 0117U003920) of Zaporizhzhia National Technical University.