The target of my research program is to nd new approaches to train Neural Networks (NN), in particular using Evolutionary Algorithms (EA). This area of research, called Neuroevolution, studies the optimization of topology and weights of neural networks. The EAs are meta-algorithms, which have many advantages with respect to classical optimization techniques based on gradient descent (GD): (i) the loss function does not need to be di erentiable, (ii) they do less likely get stuck in local minima, (iii) they are easy to parallelize. The rst approach was the application of Di erential Evolution (DE) to NNs for supervised problems, and the results were promising. Moreover, the advantages abovementioned are the reasons why the EAs are largely used in other machine learning task, like Reinforcement Learning. Consequently, the application of EAs as NNs optimizer for Reinforcement Learning will be my future research subject.
Supervised Learning is a Machine Learning task where a function is inferred from examples composed of a pair of input and corresponding output. To tackle this task many models were proposed. Among the most important model is the Arti cial Neural Network (ANN), which tries to emulate the human brain behaviour through a weighted graph.
The usual method used to train the weights of Neural Network is the Backpropagation (BP) with stochastic gradient descent (SGD), which minimizes the loss function that evaluates the error between the NN outputs and the expected outputs. The minimization is achieved through the modi cation of the Neural Network weights. BD with SGD is an iterative local-convergence method which needs a loss function di erentiable, and it is hard to parallelize.
There are other ways to train a Neural Network; one of them is the use of meta-algorithms coming from the Neuroevolution eld. There are many advantages in using EAs. First of all, they can optimize functions which are not di erentiable, since no derivatives are calculated by these algorithms. Then, EAs are less likely to get stuck in local minima. They use a pool of solutions, combined with greedy methods, in order to create new solutions which explore most of the search space. Finally, EAs are easy to parallelize. In fact, they usually use pointwise operators, which can be concurrently applied to di erent solutions belonging to the same pool. However, there are some disadvantages, in fact, EAs su ers from stagnation problem and they are slower to converge in a local minimum.
With the aim to exploit the advantages of EAs, my research program has focused on the studying of the Di erential Evolution (DE). DE is an Evolutionary Algorithm, which is used to optimize real-valued functions without the gradient. The DE is a composition of four meta-operations: Initialization, Mutation, Crossover, Selection. For each operator, there are many variants developed in order to improve DE performances. Moreover, the DE has two hyperparameters, the rst one is F used by Mutation operator, and the other one is the used by Crossover operator. These parameters are essential for reaching the convergence. For this reason, there exist many self-adaptive methods of the DE hyperparameters.
The idea is to use DE as an optimizer of Deep Neural Networks. So, a batching system was proposed in order to reduce the computation time. Moreover, in order to nd the best combination of the operators (and their parameters), they have been tuned through a Quade test analysis on several datasets.
Another application of NNs is Algorithm Learning, i.e the Machine Learning
task where an algorithm is inferred from examples. The rst model proposed by
Alex Graves at. al. is called Neural Turing Machine (NTM) [
The common way for the training a NN in Supervised Learning approach is the
Backpropagation with Stochastic Gradient [
On the other hand, Neuroevolution is not very common for business application. This is the reason why using EAs as NNs optimizer is a problem which lacks a properly de ned state-of-the-art.
Anyway, the dominating approach used in Neuroevolution is the genetic one
[
Another way is the use of Evolutionary Algorithms which work on real values
such as DE. There are many papers on the application of DE as an optimizer
for small NNs. In [
Another Machine learning eld, where Neuroevolution is growing up, is the
Reinforcement Learning [
In my master thesis, I proposed the design and the development of a machine
learning method based on DE for neural network optimization: DENN (Di
erential Evolution for Neural Networks). For DENN, I developed a batching system
with an e ective tness inheritance method, which permitted the use of DE as a
Feedforward Neural Networks optimizer for real Supervised classi cation
problems. The algorithm and the related tests are presented in [
After the thesis, I continued to study and develop variants of DE in order to
improve DENN. During my rst Ph.D. year, I focused on trying all the common
variants of DE and its operators in order to determine which of them was the
best for NNs training. Moreover, during this phase, I developed a new Crossover
operator, called interm, which has proven to work well. The operators (and their
parameters) have been tuned through a Quade test analysis performed on several
datasets. The results of the analysis and the interm implementation are shown
in [
Subsequently, I worked on the application of DENN on Machine Learning
tasks where the classic methods do not perform well; I developed a variant of
DENN which uses a model based on Neural Turing Machines (NTM) [
Some of the objectives of my proposal were already achieved: rst, I designed and developed a DE variant which is able to train a Feedforward NN; then, I tuned the DENN operations and their parameters in order to nd the best combination to be applied to Supervised Learning problems. Finally, I used DENN as optimizer of Feedforward Neural Networks for Algorithm learning. For the future, the long-term objectives of my proposal can be divided into three steps: the rst one is to use the knowledge I acquired on DE in order to design Evolutionary Algorithms able to train Recurrent Neural Networks (RNN). The next step will be to apply Neuroevolution techniques to train a Recurrent Neural Controller for a NTM-like model. Moreover, I am planning to design and develop EAs variants to be applied to Reinforcement learning problems. In the beginning, I would like to use EAs in virtual worlds, such as the Starcraft II Learning Environment, the environment used by Google DeepMind to improve Starcraft 2's AI. Such environments are good to train and test models because of their controlled structure.
I applied evolutionary algorithms to the Feedforward Neural Networks
optimization, with good results presented in [
About the results, on the small dataset DENN reaches better results then Backpropagation.
For the known MNIST dataset, the DENN algorithm applied to a network without the hidden layers, reaches more than 90% of accuracy, the same result obtained by Backpropagation.
The Quade test analysis identi ed the best combination of DE operators, which are current to pbest as Mutation, interm as Crossover, and SHADE as Self-Adaptive method.
Regarding NRAM and the problem of algorithms learning, the preliminary
results will be presented at AI*IA 2018 [
During the last years, Machine learning became popular not only for the research world, but also in the business eld.
Looking at both preliminary results obtained during rst year and others recent researches, the study of Evolutionary Algorithms on deep learning problems appears to be promising in order to solve real world problems.
In the future, more attention will be given to the research area of Reinforcement learning, which can guarantee a more general approach with respect to Supervised learning.