The introduction of artificial neural networks in the process of medical diagnosis meets a number of difficulties. The using of general methods for synthesis and training networks is difficult because of the complexity of the simulated system, that is the human. Neuroevolution approach for the synthesis of neural network models has proven itself well, but it is also not without difficulties. The paper proposes a new mechanism for modifying the genetic algorithm in the synthesis of neuromodels that can be used in medical diagnostics. Innovations allow to reduce time of synthesis, and also to solve a number of problems at a choice of the best individuals for formation of new population.
In the arsenal of modern medicine, there are many effective means of detecting a variety of diseases, but some of them are invasive, dangerous to the patient or difficult to implement and high-cost. Most of these techniques can afford only multicommercial medical centers, and therefore inaccessible to the majority of population. Modern medicine, especially at the primary level, needs to be armed with inexpensive, safe for the patient, effective and reliable tools for the earliest possible detection of the most common forms of pathology. One of the ways to create such tools is the use of artificial neural network (ANN) technologies.
Neural networks are implemented according to the principles of construction and functioning of the human brain. From it that technologies inheritance the ability to learn and extract knowledge from statistical data, to generalize them in the form of rules and regularities, the property of intuition. Well-designed and properly trained ANN are able to build adequate mathematical models and use them to perform highprecision predictions (forecasts in many areas, including medicine.
ANNs are not programmed in the usual sense of the word, they are trained. The possibility of training is one of their main advantages over traditional algorithms. After training, ANNs become mathematical models of the subject areas under consideration. This means that virtual experiments can be performed on them, and ANNs will behave in exactly the same way as the subject area they are modeling.
The method of mathematical modeling in its classical sense has long been fruitfully used in many scientific fields. Today, no sufficiently complex technical object or process is created and launched without virtual computer experiments being performed on its mathematical model. Thanks to this, scientists and engineers know exactly how long the object they create will live, how it will behave in complex changing conditions, and what should be done to avoid trouble.
Experts note that the method of mathematical modeling for a long time was practically unavailable for use in the field of medical sciences due to the exceptional complexity of the object of modeling the human. But new ANN technologies allow to overcome this barrier and to construct mathematical models of patients and to carry out computer experiments on them: changing a way of life virtually, trying various courses of treatment, selecting medicines and observing on the computer screen to what it will lead.
Moreover, as the scientists note, cases have been repeatedly recorded when in the
process of neural network modeling new, previously unknown knowledge and
patterns were revealed and used. The results of neural network modeling-diagnoses and
forecasts, eventually found confirmation, despite the apparent paradoxical nature of
the detected patterns [
The facts discovered by the method of neural network mathematical modeling are
not always consistent with the established practice of giving the same
recommendations to all patients without exception: to follow a diet, abandon bad habits, limit the
use of coffee and alcohol, lose weight, limit mental and physical activity, etc [
However, there are a number of difficulties in the union of ANN technology and health protection. Decision issued using ANN must to be accompany acceptable explanations and comments that ANN are not able to do because of their not verbality inherited from the prototype brain. Moreover, the theory of neural networks is still weak and only a very experienced mathematician can create a really adequate ANN model that provides high accuracy of diagnosis and forecasting.
To solve this problem, often turned to strategies that allow not just to train, but to
synthesize ANN in the same way as it happens in the real world these are
neuroevolutionary methods. However, it should be emphasized that in this case, the developer
instead of difficulties with the development and training of ANNs gets problems with
the use of evolutionary algorithms. In this research, authors consider the
modernization of the previously proposed modified genetic algorithm [
Inasmuch the choice of ANN topology is, as a rule, a complex task solved by trial and
error, the evolutionary search for a neural network structure is able to facilitate and to
some extent automate the process of solving the problem of configuring and training
ANNs. The simultaneous solution of two separate problems: setting the weights of
connections and setting the structure of ANN allows to some extent compensate for
the shortcomings inherent in each of them separately and combine their advantages.
On the other hand, the payment for this is a huge search area, as well as the
unification of a number of shortcomings caused by the use of the evolutionary approach.
Summing up, let list the advantages and disadvantages [
Advantages [
ANN model by considering the non-standard, irregular topologies; ─ independence from the structure of ANN and characteristics of activation functions of neurons; ─ the ability to automatically search the topology of the ANN and obtain a more accurate ANN model.
To simplify the task and improve the quality of the results, in the process of
searching for the topology of ANN, it is possible to use additional regulatory restrictions
that help to avoid excessive complication of the network, which is expressed in a
rapid increase in the number of hidden neurons and connections between them [
Disadvantages [
The first drawback is the main problem of evolutionary tuning of the ANN
structure. It is mainly due to the sensitivity of training results to initial conditions and
values of training algorithm parameters [
Using of selective and crossover operators during the synthesis of
neuromodels
Selection of individuals consists in the selection (for the value of the fitness function
calculated at the previous stage) of those individuals who will participate in the
breeding of children for the next population, that is, for the next generation. This choice is
where
made according to the principle of natural selection, according to which the
chromosomes with the highest values of fitness function have the greatest chances to
participate in the creation of new individuals [
F Indi , (1) and F Indi is a value of the fitness function of the individual Ind i , and ps Indi is a probability of selection of individuals Ind i . Selection of an individual can be represented as the result of turning the roulette wheel, since the selected individual (that is, the winner) refers to the sector of the wheel that fell out. Obviously, the larger the sector, the greater the likelihood of selecting the appropriate individual. Therefore, the probability of choosing this chromosome is proportional to the value of its fitness function.
The roulette wheel selection method is considered by genetic algorithms to be the
main method of selecting individuals for the parent population with a view to their
subsequent transformation by genetic operators, such as crossing and mutation [
pool is the number of copies, which is set by the expression ps Indi n . F Indi i1 Each individual gets in the parent pool is the number of copies, which is set by the expression cIndi ps Indi n , (2) where n are the number of the individuals Ind i for i 1,2,...,n in the population, and ps Indi is a probability of selection of an individual Ind i , what is calculated by
ps Indi n . Strictly speaking, the number of copies of a given individual F Indi i in the parent pool is equal to an integer part of cIndi . When using formulas (1) and (2) it is necessary to pay attention to the fact that cIndi F Indi , where F is the F average value of the fitness function in the population. Obviously, the roulette method can be used when the value of the fitness function is positive. This method can only be used in function maximization problems (not minimization).
At tournament selection all individuals of population are divided into subgroups
with the further choice in each of them of an individual with the best fitness [
The tournament method is suitable for solving problems of both maximization and minimization of the function. In addition, it can be easily extended to problems related to multi-criteria optimization, that is, to the case of simultaneous optimization of several functions. In the tournament method, it can be changed the size of the subgroups into which the population is divided (tournament size). Studies confirm that the tournament method is more effective than the roulette method.
During ranking selection individuals of the population are ranked according to the values of their fitness function. This can be thought of as a sorted list of individuals, ordered in the direction from the most adapted to the least adapted (or vice versa), in which each individual is assigned a number, which determines its place in the list and is called a rank. The number of copies of each individual introduced into the parent population is calculated by a priori given function depending on the rank of the individual.
The advantage of the rank method is that it can be used both to maximize and
minimize the function. It also does not require scaling due to the problem of premature
convergence relevant to the roulette method [
The application of genetic operators to individuals selected by selection leads to the formation of a new population of children from the parent population created at the previous stage.
The crossover operation consists in the exchange of fragments of chains between
two parent individuals [
Two-point crossover, as its name implies, differs from point crossing in that
descendants inherit fragments of parent individuals determined by two randomly
selected crossing points [
Multiple-point crossover is a generalization of previous operations and is
characterized by a correspondingly large number of crossing points [
Uniform crossover is performed according to a randomly selected standard that
specifies which genes should be inherited from the first father (other genes are taken
from the second parent) [
g Ind3 {g1 Randg Ind1 , g Ind2 , .
(3) An example of uniform crossover is shown in Fig. 3.
parent1: 0011001110 10 crossover 1011011110 10: child1 parent 2 : 1010110110 11 0010100110 11: child 2 locus : 1 2 3 4 5 6 7 8 9 10 11 12 etalon : 0 1 0 1 1 0 1 1 1 0 1 1 The main factor of evolution is natural selection, which leads to the fact that among genetically different individuals of the same population survive and leave offspring only the most adapted to the environment. In genetic algorithms is also highlighted stage of selection, which from the current population are selected and included in the parent population of individuals who have the greatest value of a fitness function. The next step, sometimes called evolution, involves genetic crossover and mutation operators that recombine genes on chromosomes.
However, it should be recognized that the classical genetic algorithm emulates natural evolution is not fully, so it may exist wish to look at the mechanism of selective pressure, which will more effectively perform crossover.
We introduce selective pressure at the crossover stage by extending the selection operation. It is established the relationship between the probability of gene transmission to a descendant and the knowledge of the suitability of parents. To do this, we will expand the rank selection by introducing additional criteria for evaluating individuals.
The first criterion will be used to assess memory redundancy. As mentioned above,
neural networks have memory that implemens as the weights connections. The less
memory ANN has, the fewer images it can remember. However, in a situation where
two ANNs with different memory provide the necessary accuracy of recognition
(evaluation), the network with less memory, of course, has the best generalizing
properties [
WFFc WFBc samp Inst sampFeat , (4) where WFFc is the number of direct ANN connections (WFFc w1, w2 ,...,wi );WFBc is the number of feedback ANN connections (WFBc w1, w2 ,...,wj ); samp Inst is the number of instances at training set; samp Feat is the number of features at training set.
If critm 1 , then the ANN memory is redundant (the ANN memory dimension is greater than the sample size). If critm 1 , then the ANN can remember the entire training sample (the memory dimension of the ANN is equal to the size of the training sample). If critm 1 , then the ANN will not be able to remember exactly the entire training sample (the memory dimension of the ANN is less than the dimension of the training sample), but the ANN will show generalizing and approximating abilities.
The use of the second criterion is related to the approximation properties of the
ANN. One of the most important characteristics of ANN models is the quality of
approximation. In the case where the error level of the models is one-to-one, the
approximation quality is higher in the model that uses fewer links [
Error WFFc WFBc Ww0 where Error is the aggregate error allowed by the network (for example, root mean square error) is such that Error , where is the maximum allowable error (learning objective). As an Error it can be used a sample error ( Eнав. ) or a test sample error as an error ( Eтест. ); WFFc is the number of direct connections of the ANN ( WFFc w1, w2 ,...,wi ); WFBc is the number of feedbacks for recurrent ANN ( WFBc w1, w2 ,...,wj ); Ww0 are zero weights (ANN connections whose weight is
Thus, we consider the modification of rank selection using criteria for evaluating ANN moles.
Selection begins by sorting (ranking) individuals based on their availability so that FIndi FInd j for i j . Each individual is then assigned a probability of being selected ps , taken from a given restricted division ps 1 . The probability of sei lection is calculated by the form: psi
a a b rank crit m crit a 1 1
, n n 1 (6) where a 1;2, b 2 a , rank is a rank of individuals in the sorted list of individuals.
The use of criteria in determining the probability of selection solves several problems, namely: ─ advance convergence of the method; ─ reducing the variety; ─ selection of the best individuals, at the same rank (at the same value of the fitness function).
It has long been known that setting the probability of transmission of the parent gene to the offspring in uniform crossing can significantly increase its efficiency, and also allows to emulate other crossover operators (single-point, two-point). It is also known that the use of a uniform crossing operator allows to apply the so-called multiparent recombination, when to cross one child more than two parents. Uniform crossing gives greater flexibility when combining strings, which is an important advantage when working with genetic algorithms.
Therefore, a uniform crossing with a specified parent pool size will be used as the crossing operator. The pool will be filled with individuals selected using modified rank selection. This approach adds flexibility to the method and allows to hope for a change in the behavior of the method. 4
Data for testing were taken from the open repository – UC Irvine Machine Learning
Repository. Data sample was used: Breast Cancer Coimbra Data Set [
The relative error value in this case will be calculated as the ratio of the classification error to the total sample size (number of instances).
E
errorclass
100% , (7) where E is relative error; errorclass is classification error; Numbersampl the number of instances in the sample.
The following hardware and software have been used for experimental verification of the proposed method for ANN synthesis: the computing system of the Department of software tools of Zaporizhzhia Polytechnic National University (ZPNU), Zaporizhzhya: Xeon processor E5-2660 v4 (14 cores), RAM 4x16 GB DDR4, the programming model of Java threads. 5
From the results of the experiment it can be seen that the most acceptable performance of the method is observed at the size of the parent pool 2 and 7. In other cases of parents, the ratio of resources used and time spent in the exact initial neuromodels is not satisfactory. It can also be noted that the number of parent individuals for crossing >10 does not make sense, because with large values of the execution time and the overhead of sending data, the accuracy deteriorates significantly.
Moreover, it can be concluded that the use of selective pressure and uniform
crossing reduce the size of the population, without taking into account and without
considering those individuals of the population that are characterized by a small value of the
fitness function. Also, selective pressure allows to take into account additional quality
indicators of neural network models [
The increasing of accuracy and reduction of memory and computing power costs for storing and crossing the total population volume confirm the high efficiency of the proposed modification. However, the growth of input parameters should be noted. Therefore, the next step may be to automate the selection of input parameters, depending on the problem and its boundaries.