=Paper=
{{Paper
|id=Vol-2640/paper_1
|storemode=property
|title=Evolving Robust Neural Architectures to Defend from Adversarial Attacks
|pdfUrl=https://ceur-ws.org/Vol-2640/paper_1.pdf
|volume=Vol-2640
|authors=Shashank Kotyan,Danilo Vasconcellos Vargas
|dblpUrl=https://dblp.org/rec/conf/ijcai/KotyanV20
}}
==Evolving Robust Neural Architectures to Defend from Adversarial Attacks==
https://ceur-ws.org/Vol-2640/paper_1.pdf
Evolving Robust Neural Architectures to Defend from Adversarial Attacks
Shashank Kotyan and Danilo Vasconcellos Vargas
Department of Informatics, Kyushu University, Japan
shashankkotyan@gmail.com and vargas@inf.kyushu-u.ac.jp
Abstract Regarding NAS, the automatic design of architectures has
been of broad interest for many years. The aim is to develop
Neural networks are prone to misclassify slightly
methods that do not need specialists in order to be applied to
modified input images. Recently, many defences
a different application. This would confer not only general-
have been proposed, but none have improved the
ity but also easy of use. Most of the algorithms for NAS are
robustness of neural networks consistently. Here,
either based on reinforcement learning [Pham et al., 2018;
we propose to use adversarial attacks as a function
Zoph et al., 2018] or evolutionary computation [Real et al.,
evaluation to search for neural architectures that
2017; Miikkulainen et al., 2019]. On the one hand, in re-
can resist such attacks automatically. Experiments
inforcement learning approaches, architectures are created
on neural architecture search algorithms from the
from a sequence of actions which are afterwards rewarded
literature show that although accurate, they are not
proportionally to the crafted architecture’s accuracy. On
able to find robust architectures. A significant rea-
the other hand, in evolutionary computation based methods,
son for this lies in their limited search space. By
small changes in the architecture (mutations) and recombi-
creating a novel neural architecture search with op-
nations (crossover) are used to create new architectures. All
tions for dense layers to connect with convolution
architectures evolved are evaluated based on their accuracy.
layers and vice-versa as well as the addition of con-
Some of the best architectures based on this accuracy are cho-
catenation layers in the search, we were able to
sen to continue to the next generation.
evolve an architecture that is inherently accurate on
adversarial samples. Interestingly, this inherent ro- Here we propose the use of NAS to tackle the robustness
bustness of the evolved architecture rivals state-of- issues exposed by adversarial samples. In other words, ar-
the-art defences such as adversarial training while chitecture search will be employed not only to find accurate
being trained only on the non-adversarial samples. neural networks but also robust ones. This is based on the
Moreover, the evolved architecture makes use of principle that robustness of neural networks can be assessed
some peculiar traits which might be useful for de- by using accuracy on adversarial samples as an evaluation
veloping even more robust ones. Thus, the results function. We hypothesise that if there is a solution in a given
here confirm that more robust architectures exist as architecture search space, the search algorithm would be able
well as opens up a new realm of feasibilities for the to find it. This is not only a blind search for a cure. The best
development and exploration of neural networks. architectures found should also hint which structures and pro-
cedures provide robustness for neural networks. Therefore, it
would be possible to use the results of the search to under-
1 Introduction stand further how to improve the representation of models as
Neural Architecture Search (NAS) and adversarial samples well as design yet more robust ones.
have rarely appeared together. Regarding adversarial sam-
ples, they were discovered in 2013 when neural networks
were shown to behave strangely for nearly the same im- 2 Adversarial Machine Learning
ages [Szegedy, 2014]. Afterwards, a series of vulnerabili- Adversarial machine learning is a constrained optimisation
ties were found in [Moosavi-Dezfooli et al., 2017; Su et al., problem. Let f (x) ∈ [[1..N ]] be the output of a machine
2019]. Such adversarial attacks can also be easily applied to learning algorithm in multi-label classification setting. Here,
real-world scenarios which confer a big problem for current x ∈ Rk is the input of the algorithm for the input of size k
deep neural networks’ applications. Currently, there is not and N is the number of classes in which x can be classified.
any known learning algorithm or procedure that can defend In Image Classification problem k = m × n × 3 where m × n
against adversarial attacks consistently. is the the size of the image. Adversarial samples x0 can be
Copyright c 2020 for this paper by its authors. Use permitted thus defined as follows:
under Creative Commons License Attribution 4.0 International (CC
BY 4.0). x 0 = x + x such that f (x0 ) 6= f (x) (1)
in which x ∈ Rk is a small perturbation added to the input. Search, Bayesian Optimization, Evolutionary Methods, Re-
Therefore, adversarial machine learning can be defined as an inforcement Learning, and Gradient Based Methods. Finally,
optimization problem1 : a performance estimation (usually error rate) is required to
evaluate the explored architectures.
minimize g(x + x )c subject to kx k ≤ th (2) Currently, most of the current NAS suffer from high com-
x
putational cost while searching in a relatively small search
where th is a pre-defined threshold value and g()c is the soft- space [Lee et al., 2018; Lima and Pozo, 2019].It is already
label or confidence for the correct class c such that f (x) = shown in [Yu and Kim, 2018] that, if there is a possibility of
argmax g(x) fitness approximation at small search spaces, we could evolve
Moreover, attacks can be divided according to the func- algorithms in an ample search space. Moreover, many ar-
tion optimised. In this way, there are L0 (limited number of chitecture searches focus primarily on the hyper-parameter
pixels attacked), L1 , L2 and L∞ (limited amount of varia- search while using architecture search spaces around previ-
tion in each pixel) types of attacks. There are many types ously hand-crafted architecture [Lee et al., 2018; He et al.,
of attacks as well as their improvements. Universal per- 2019] such as DenseNet which are proved to be vulnerable to
turbation types of attacks were shown possible in which a adversarial attacks [Vargas and Kotyan, 2019]. Therefore, for
single perturbation added to most of the samples is capable finding robust architectures, it is crucial to expand the search
of fooling a neural network in most of the cases [Moosavi- space beyond the current NAS.
Dezfooli et al., 2017]. Moreover, extreme attacks such as SMASH [Brock et al., 2017] uses a neural network to gen-
only modifying one pixel (L0 = 1) called one-pixel attack erate the weights of the primary model. The main strength
is also shown to be surprisingly effective [Su et al., 2019; of this approach lies in preventing high computational cost,
Vargas and Kotyan, 2019]. Most of these attacks canbe eas- which is incurred in other searches. However, this comes at
ily transferred to real scenarios by using printed out versions the cost of not being able of tweaking hyperparameters which
of them [Kurakin et al., 2016]. Moreover, carefully crafted affect weights like initialisers and regularisers. Deep Archi-
glasses [Sharif et al., 2016] or even general 3D adversarial tect [Negrinho and Gordon, 2017] follows a hierarchical ap-
objects are also capable of causing misclassification [Atha- proach using various search algorithms such as Monte Carlo
lye and Sutskever, 2018]. Regarding understanding the phe- Tree Search (MCTS) and Sequential Model based Global Op-
nomenon, it is argued in [Goodfellow et al., 2014] that neural timization (SMBO).
networks’ linearity is one of the main reasons. Another re-
cent investigation proposes the conflicting saliency added by
adversarial samples as the reason for misclassification [Var- 4 Searching for Robust Architectures
gas and Su, 2019]. A robust evaluation (defined in Section 4.1) and search algo-
Many defensive systems were proposed to mitigate some rithm must be defined to search for robust architectures. The
of the problems. However, current solutions are still far search algorithm may be a NAS provided that some modifi-
from solving the problems. Defensive distillation [Paper- cations are made (Section 4.2). However, to allow for a more
not et al., 2016] uses a smaller neural network to learn the extensive search space, which is better suited to the problem,
content from the original one; however, it was shown not to we also propose the Robust Architecture Search (Section 5).
be robust enough [Carlini and Wagner, 2017]. The addition
of adversarial samples to the training dataset, called adver- 4.1 Robustness Evaluation
sarial training, was also proposed [Goodfellow et al., 2014; Adversarial accuracy may seem like a natural evaluation
Huang et al., 2015; Madry et al., 2018]. However, adversar- function for assessing neural networks’ robustness. However,
ial training has a strong bias in the type of adversarial sam- there are many types of perturbations possible; each will re-
ples used and is still vulnerable to attacks Many recent varia- sult in a different type of robustness assessment and evolu-
tions of defences were proposed which are carefully analysed, tion. For example, let us suppose an evaluation with th = 5
and many of their shortcomings explained in [Athalye et al., is chosen, robust networks against th = 5 might be devel-
2018; Uesato et al., 2018]. In this article, different from pre- oped. At the same time, nothing can be said for other th and
vious approaches, we aim to tackle the robustness problems attack types (different L). Therefore, th plays a role but the
of neural networks by automatically searching for inherent different types of L0 , L1 , L2 and L∞ completely change the
robust architectures. type of robustness, such as wide perturbations (L∞ ), punc-
tual perturbations (L0 ) and a mix of both (L1 and L2 ). To
3 Neural Architecture Search avoid creating neural networks that are only robust against
There are three components to a neural architecture search: one type of robustness and at the same time to allow robust-
search space, search strategy and performance estimation. A ness to slowly build-up from any partial robustness, a set of
search space substantially limits the representation of the ar- adversarial attacks for varying th and L are necessary.
chitecture in a given space. A search strategy must be em- To evaluate the robustness of architectures in varying th
ployed to search for architectures in a defined search space. and L while at the same time keeping computational cost low,
Some widely used search strategies for NAS are: Random we use here a transferable type of attack. In other words, ad-
versarial samples previously found by attacking other meth-
1 ods are stored and used as possible adversarial samples to the
Here the definition will only concern untargeted attacks but a
similar optimization problem can be defined for targeted attacks current model under evaluation. This solves the problem that
Layer Population Block Population Model Population
Layer1 Block1 Layer1 Model1 Block3
Layer16
Layer2 Block2 Model2
Layer4 Block5
Layer5 Block5
Layer50 Block50 Model25
Figure 1: Illustration of the proposed RAS structure with three subpopulations.
L0 Attack L∞ Attack
Model Attack Optimiser Total
th = 1 th = 3 th = 5 th = 10 th = 1 th = 3 th = 5 th = 10
DE 18 46 45 47 05 09 12 24 206
CapsNet
CMA-ES 14 34 45 62 09 38 74 98 374
DE 23 66 75 77 06 22 46 78 393
ResNet
CMA-ES 11 49 63 77 28 72 75 83 458
DE 23 59 63 66 00 02 03 06 222
AT
CMA-ES 20 50 70 82 03 12 25 57 319
DE 21 73 78 78 04 21 45 78 398
FS
CMA-ES 17 49 69 78 26 63 66 74 442
Total 147 426 508 567 81 239 346 498 2812
Table 1: The number of samples used from each type of black-box attack to compose the 2812 adversarial samples. Based on the principle
of the transferability of adversarial samples, these adversarial samples are used as a fast attack for the robustness evaluation of architectures.
Details of the attacks as well as the motivation for using a model-agnostic (black-box) dual quality (L0 and L∞ ) assessment are explained in
detail at [Vargas and Kotyan, 2019].
most of the attacks are usually slow to be put inside a loop the search for robustness and accuracy. Here we use SMASH
which can make the search for architectures too expensive. and DeepArchitect for the tests. The reason for the choice
Table 1 shows a summary of the number of images used lies in the difference between the methods and availability of
from each type of attack, totalling 2812 adversarial samples. the code. Both methods have their evaluation function modi-
Samples were generated using the model agnostic dual qual- fied to contain not only accuracy but also robustness (Section
ity assessment [Vargas and Kotyan, 2019]. Specifically, we 4.1).
use the adversarial samples from two types of attacks (L0 and
L∞ attacks) with two optimization algorithms (Covariance 5 Robust Architecture Search (RAS)
Matrix Adaptation Evolution Strategy (CMA-ES) [Hansen et
al., 2003] and Differential Evolution (DE) [Storn and Price, Here, we propose an evolutionary algorithm to search for ro-
1997]). We use CIFAR-10 dataset [Krizhevsky et al., 2009] bust architectures called Robust Architecture Search (RAS)2 .
to generate the adversarial samples. We attacked traditional It makes sense to focus here on search spaces that allow for
architectures such as ResNet [He et al., 2016] and CapsNet unusual layer types and their combinations to happen, which
[Sabour et al., 2017]. We also attacked some state-of-art- is more vast than the current traditional search spaces. The
defences such as Adversarial Training (AT) [Madry et al., motivation to consider this vast search space is that some
2018] and Feature Squeezing (FS) [Xu et al., 2017] defending of the most robust architectures might contain these unusual
ResNet. The evaluation procedure consists of calculating the combinations which are not yet found or deeply explored.
amount of successful adversarial samples divided by the total RAS Overview RAS works by creating three initial popu-
of possible adversarial samples. This also avoids problems lations (layer, block and model populations). Every genera-
with different amount of perturbation necessary for attacks to tion, the model population have each of its members modified
succeed, which could cause incomparable results. five times by mutations. The modified members are added to
the population as new members. Here we propose a utility
4.2 Robust Search Conversion of existing NAS evaluation in which layer and block populations are evaluated
By changing the fitness function (in the case of evolution- by the number of models (architectures) using them. Models
ary computation based NAS) or the reward function (in the are evaluated by their accuracy and attack resilience (accu-
case of reinforcement learning based NAS), it is possible to racy on adversarial samples). All blocks and layers which are
create robust search versions of NAS algorithms. In other
2
words, it is possible to convert the search for accuracy into Code is available at http://bit.ly/RobustArchitectureSearch
Cluster Population Child Population
Mutations
Step 1
Model1 Model1_1 Model1_2
Closest Individual to
Modeli based on
Spectrum Individual
Model2 Step 3
Modelc Modeli Step 2 Model2_1 Model2_2
Fitness
Evaluation Model25_1
Model25 Model25_2
Better Fitness Individual
Step 4
Figure 2: Illustration of the proposed Evolutionary Strategy. In this strategy, Step 2, 3, and 4 are repeated for all the individuals in the child
population. Step 1 is repeated when all the individuals in the child population have been evaluated against a cluster individual.
not used by any of the current members of the model popu- Block Mutation Block mutation change a single block in
lation are removed at the end step of each generation. More- the block population. The possibilities are: e) Add Layer: A
over, architectures compete with similar ones in their sub- random layer is added to a chosen random block, f) Remove
population, such that only the fittest of each subpopulation Layer: A random layer is removed from a chosen random
survives. block, g) Add Layer Connection: A random connection be-
tween two layers from the chosen random block is added, h)
5.1 Description of Population Remove Layer Connection: A random connection between
For such a vast search space to be more efficiently searched, the two layers from the chosen random block is removed, i)
we propose to use three subpopulations, allowing for the Swap Block: Chosen block is swapped with a random block
reuse of blocks and layers. Specifically, the layers consist of: from the population.
Layer Population: Raw layers (convolutional and fully con- Model Mutation Model mutation modify a given architec-
nected) which make up the blocks. Block Population: Blocks ture. The possible model mutations are: j) Add Block: A ran-
which are a combination of layers. Model Population: A dom block is added to the model, k) Remove Block: A random
population of architectures which consists of interconnected block is removed from the model, l) Add Block connection: A
blocks. Figure 1 illustrates the architecture. random connection between the two blocks is added, m) Re-
The initial population consists of 25 random architectures move Block connection: A random connection between the
which contain U (2, 5) blocks made up of U (2, 5) layers, in two blocks is removed.
which U (a, b) is a uniform random distribution with mini- All mutations add a new member to the population instead
mum a and maximum b values. The possible available pa- of substituting the previous one. In this manner, if nothing
rameters for the layers are as follows: for convolutional lay- is done, the population of layers and blocks may explode, in-
ers, filter size might be 8, 16, 32 or 64, stride size may be 1 creasing the number of lesser quality layers and blocks. This
or 2 and kernel size is either 1, 3 or 5; for fully connected would cause the probability of choosing functional layers and
layers, the unit size may assume the values of 64, 128, 256 blocks to decrease. To avoid this, when the layer or block
or 512. All the layers use Rectified Linear Unit (ReLU) as an population exceeds 100 individuals, the only layer/block mu-
activation function and are followed by a batch-normalisation tation available is swap layers/blocks.
layer.
5.3 Objective (Fitness) Function
5.2 Mutation Operators Fitness of an individual of the model population is measured
using the final validation accuracy of the model after training
Regarding the mutation operators used to evolve the architec-
for a maximum of 200 epochs with early stopping if accu-
ture, they can be divided into layer, block and model muta-
racy or validation accuracy do not change more than 0.001 in
tions which can only be applied to the respective layer, block
the span of 15 epochs. Regarding the fitness calculation, the
and model populations’ individuals. The following para-
fitness is calculated as the accuracy of the model plus the ro-
graphs define the possible mutations.
bustness of the model (Fitness = Accuracy + Robustness).
Layer Mutation Layer mutations are of the following The Accuracy of the architecture is calculated after the
types: a) Change Kernel: Changes the kernel size of the con- model is trained for 50 epochs over the whole set of sam-
volution layer, b) Change Filter: Changes the filter size of ples (50000 samples) of the CIFAR-10’s training dataset for
the convolution layer, c) Change Units: Changes the unit size every 10 generation(s) or over 1000 random samples of the
of the fully connected layer, d) Swap Layer: Chosen layer is CIFAR-10 training dataset for all other generations. This al-
swapped with a random layer from the layer population. lows an efficient evolution to happen in which blocks and lay-
ers evolve at a faster rate without interfering with the archi- Here, experiments are conducted on both the proposed
tecture’s accuracy. Using entire dataset subjects to evolving RAS and converted versions of DeepArchitect and SMASH.
the architecture to have better accuracy and using a subset of The objective is to achieve the highest robustness possible
the dataset evolves the layers and blocks of the architecture at using different types of architecture search algorithms and
a faster rate. The Robustness of the architecture is calculated compare their result and effectiveness. Initially, DeepArchi-
using accuracy on adversarial samples as described in Section tect and Smash found architectures which had an error rate
4.1. of 11% and 4% respectively when the fitness is only based
on the neural network’s testing accuracy. However, when the
5.4 Spectrum-based Niching Scheme accuracy on adversarial samples is included in the evaluation
function, the final error rate increases to 25% and 23% re-
To keep a high amount of diversity while searching in a vast
spectively (Table 2). This may also indicate that poisoning
search space by using a novel algorithm described below also
the dataset might cause a substantial decrease in accuracy for
shown in Figure 2. This niching scheme uses the idea of
the architectures found by SMASH and DeepArchitect. In
Spectrum-based niching from [Vargas and Murata, 2017] but
the case of RAS, even with a more extensive search space, an
explores a different approach to it. First, all the initial pop-
error rate of 18% is achieved.
ulation is converted into a cluster population such that each
individual in the initial population is a cluster representative. Regarding the robustness of the architectures found, Table
Then we create two child individuals for each cluster rep- 2 shows that the final architecture found by DeepArchitect
resentative by randomly applying five mutation operators on and SMASH were very susceptible to attacks, with error rate
cluster representative. We then find the closest cluster rep- on adversarial samples of 75% and 82% respectively. De-
resentative to the child individual using spectrum described spite the inclusion of the R (measured accuracy on adver-
below. If the fitness of the child individual is better than sarial samples) on the evaluation function, the architectures
the closest cluster representative than the child individual be- were still unable to find a robust architecture. This might be
comes the new cluster representative, and the old cluster rep- a consequence of the relatively small search space used and
resentative is removed from the population and the genera- more focused initialisation procedures. Moreover, the pro-
tion. The process is completed for all the individuals in a posed method (RAS) finds an architecture which has an error
cluster population. We are hence evolving a generation of the rate of only 42% on adversarial samples. Note, however,
evolution. that in the case of the evolved architecture, this is an inher-
ent property of the architecture found. The architecture is
Here, we use the spectrum as a histogram containing the
inherently robust without any kind of specialised training or
features: Number of Blocks, Number of Total Layers, Num-
defence such as adversarial training (i.e., the architecture was
ber of Block Connections, Number of Total Layer Connec-
only trained on the training dataset). The addition of defences
tions, Number of Dense Layers, Number of Convolution Lay-
should increase its robustness further.
ers, Number of Dense to Dense Connections, Number of
Dense to Convolution Connections, Number of Convolution
to Dense Connections, and Number of Convolution to Con- 7 Analyzing RAS
volution Connections. By using this Spectrum-based niching
In this section, we will evaluate the proposed architecture re-
scheme, we aim to achieve an open-ended evolution, prevent-
garding its evolution quality and how subpopulations behave
ing the evolution from converging to a single robust architec-
throughout the process. Figure 3 shows how the mean accu-
ture. Preserving diversity in the population ensures that the
racy of the architectures evolved increases over time. The pat-
exploration rate remains relatively high, allowing us to find
tern of behaviour is typical of evolutionary algorithms, show-
different architectures even after many evolution steps. For
ing that evolution is happening as expected.
the vast search space of architectures, this property is espe-
cially important, allowing the algorithm to traverse the vast In Figure 4, the overall characteristics of the evolved archi-
search space efficiently. tectures throughout the generations are shown. The average
number of blocks and the connections between them increase
over the generations. However, the average number of layers
6 Experiments on RAS and Converted NAS never reaches the same complexity as the initial models. The
number of layers decreases steeply initially while slowly in-
creasing afterwards. Therefore, the overall behaviour is that
Architecture Search Testing ER ER on Adversarial Samples blocks become smaller and numerous. A consequence of this
DeepArchitect* 25% 75% is that the number of connections becomes proportional to the
SMASH* 23% 82% number of connections between blocks and therefore exhibit
Ours 18% 42% similar behaviour. The average number of layers per block
and the average number of connections shows little change,
Table 2: Error Rate (ER) on both the testing dataset and adversar- varying only 0.5 and 0.16 respectively.
ial samples when the evaluation function has both accuracies on the Notice that the average number of layers increases but the
testing data and accuracy on the adversarial samples. average number of layers per block continues to decrease al-
*Both DeepArchitect and SMASH had their evaluation function beit slowly. Consequently, blocks tend to degenerate into a
modified to be the sum of accuracy on the testing and adversarial few layers, resulting in around three layers per block from
samples. the first average number of 3.2 layers per block. Lastly, the
Figure 3: Accuracy improvement over the generations.
Figure 4: The overall distribution of the architectures found in each generation. The connections from the input layer and the softmax layer
are always present and, therefore, they are omitted in the calculation.
average number of connections in a block is kept more or less while, in some parts of it, high-dimensional projections exist.
the same, with the mean varying throughout only from 2.1 In the whole architecture, four Dense layers in-between
to 2.26. The behaviour described above might suggest that it Convolutional ones were used, and all of the projects into
is hard to create big reusable blocks. This seems to be sup- higher dimensional space. This follows directly from Cover’s
ported by both the decrease of complexity observed as well Theorem which states that projecting into high dimensional
as the increase in the number of blocks. space makes a training set linearly separable [Cover, 1965].
Paths with Different Constraints The second peculiarity
8 Analyzing the Final Architecture: is the use of multiple paths with the different number of fil-
Searching for the Key to Inherent ters and output sizes after high-dimensional projections. No-
tice how the number of filters differs in each of the Convo-
Robustness lutional layers in these paths. This means there are different
RAS found an architecture that possesses inherent robustness constraints over the learning in each of these paths, which
capable of rivalling current defences. To investigate the rea- should foster different types of features. Therefore, this is
son behind this robustness, we can take a more in-depth look a multi-bottleneck structure forcing the learning of different
at the architecture found. Figure 5 show(s) some peculiari- sets of features which are now easily constructed from the
ties from the evolved architecture: multiple bottlenecks, pro- previous high-dimensional projection.
jections into high-dimensional space and paths with different
constraints. 9 Conclusions
Multiple Bottlenecks and Projections into High- Automatic search for robust architectures is proposed as a
Dimensional Space The first peculiarity is the use of paradigm for developing and researching robust models. This
Dense layers in-between Convolutional ones. This might paradigm is based on using adversarial attacks together with
seem like a bottleneck similar to the ones used in variational error rate as evaluation functions in NAS. Experiments on us-
autoencoders. However, it is the opposite of a bottleneck ing this paradigm with some of the current NAS had poor
(Figure 5); it is a projection in high-dimensional space. The results. This was justified by the small search space used
evolved architecture uses mostly a low number of filters by current methods. Here, we propose the RAS method,
Figure 5: Two fragments of the evolved architecture which has peculiar traits.
which has a broader search space, including concatenation, [Brock et al., 2017] Andrew Brock, Theodore Lim, James M
connections between dense to convolutional layer and vice- Ritchie, and Nick Weston. Smash: one-shot model ar-
versa. Results with RAS showed that inherently robust ar- chitecture search through hypernetworks. arXiv preprint
chitectures do indeed exist. In fact, the evolved architecture arXiv:1708.05344, 2017.
achieved robust results comparable with state-of-the-art de-
fences while not having any specialised training or defence. [Carlini and Wagner, 2017] Nicholas Carlini and David
In other words, the evolved architecture is inherently robust. Wagner. Towards evaluating the robustness of neural
Such inherent robustness could increase if adversarial train- networks. In 2017 IEEE Symposium on Security and
ing, or other types of defence, or a combination of them are Privacy (SP), pages 39–57. Ieee, 2017.
employed together with it. [Cover, 1965] Thomas M Cover. Geometrical and statistical
Moreover, investigating the reasons behind such robustness properties of systems of linear inequalities with applica-
have shown that some peculiar traits are present. The evolved tions in pattern recognition. IEEE transactions on elec-
architecture has overall a low number of filters and many bot- tronic computers, (3):326–334, 1965.
tlenecks. Multiple projections into high-dimensional space
are also present to possibly facilitate the separation of features [Goodfellow et al., 2014] Ian J Goodfellow, Jonathon
(Cover’s Theorem). It also uses multiple paths with differ- Shlens, and Christian Szegedy. Explaining and harnessing
ent constraints after the high-dimensional projection, which adversarial examples. arXiv preprint arXiv:1412.6572,
should, consequently, cause a diverse set of features to be 2014.
learned by the network. Thus, in the search space of neural
networks, more robust architectures do exist, and more re- [Hansen et al., 2003] Nikolaus Hansen, Sibylle D Müller,
search is required to find and fully document them as well as and Petros Koumoutsakos. Reducing the time complex-
their features. ity of the derandomized evolution strategy with covariance
matrix adaptation (cma-es). Evolutionary computation,
11(1):1–18, 2003.
Acknowledgments
This work was supported by JST, ACT-I Grant Number JP- [He et al., 2016] Kaiming He, Xiangyu Zhang, Shaoqing
50243 and JSPS KAKENHI Grant Number JP20241216. Ad- Ren, and Jian Sun. Deep residual learning for image recog-
ditionally, we would like to thank Prof. Junichi Murata for the nition. In Proceedings of the IEEE conference on computer
kind support without which it would not be possible to con- vision and pattern recognition, pages 770–778, 2016.
duct this research. [He et al., 2019] Xin He, Kaiyong Zhao, and Xiaowen Chu.
Automl: A survey of the state-of-the-art. arXiv preprint
References arXiv:1908.00709, 2019.
[Athalye and Sutskever, 2018] Anish Athalye and Ilya
Sutskever. Synthesizing robust adversarial examples. In [Huang et al., 2015] Ruitong Huang, Bing Xu, Dale Schuur-
Icml, 2018. mans, and Csaba Szepesvári. Learning with a strong ad-
versary. arXiv preprint arXiv:1511.03034, 2015.
[Athalye et al., 2018] Anish Athalye, Nicholas Carlini, and
David Wagner. Obfuscated gradients give a false sense of [Krizhevsky et al., 2009] Alex Krizhevsky, Geoffrey Hinton,
security: Circumventing defenses to adversarial examples. et al. Learning multiple layers of features from tiny im-
In Icml, 2018. ages. Technical report, 2009.
[Kurakin et al., 2016] Alexey Kurakin, Ian Goodfellow, and Real and stealthy attacks on state-of-the-art face recogni-
Samy Bengio. Adversarial examples in the physical world. tion. In Proceedings of the 2016 ACM SIGSAC Conference
arXiv preprint arXiv:1607.02533, 2016. on Computer and Communications Security, pages 1528–
[Lee et al., 2018] Hyeon-Chang Lee, Dong-Pil Yu, and 1540. Acm, 2016.
Yong-Hyuk Kim. On the hardness of parameter optimiza- [Storn and Price, 1997] R. Storn and K. Price. Differential
tion of convolution neural networks using genetic algo- evolution–a simple and efficient heuristic for global opti-
rithm and machine learning. In Proceedings of the Ge- mization over continuous spaces. Journal of global opti-
netic and Evolutionary Computation Conference Compan- mization, 11(4):341–359, 1997.
ion, pages 51–52, 2018. [Su et al., 2019] Jiawei Su, Danilo Vasconcellos Vargas, and
[Lima and Pozo, 2019] Ricardo HR Lima and Aurora TR Kouichi Sakurai. One pixel attack for fooling deep neural
Pozo. Evolving convolutional neural networks through networks. IEEE Transactions on Evolutionary Computa-
grammatical evolution. In Proceedings of the Genetic and tion, 23(5):828–841, 2019.
Evolutionary Computation Conference Companion, pages [Szegedy, 2014] Christian et al. Szegedy. Intriguing proper-
179–180, 2019. ties of neural networks. In In ICLR. Citeseer, 2014.
[Madry et al., 2018] Aleksander Madry, Aleksandar [Uesato et al., 2018] Jonathan Uesato, Brendan
Makelov, Ludwig Schmidt, Dimitris Tsipras, and O’Donoghue, Pushmeet Kohli, and Aaron Oord. Ad-
Adrian Vladu. Towards deep learning models resistant to versarial risk and the dangers of evaluating against
adversarial attacks. In Iclr, 2018. weak attacks. In International Conference on Machine
[Miikkulainen et al., 2019] Risto Miikkulainen, Jason Learning, pages 5032–5041, 2018.
Liang, Elliot Meyerson, Aditya Rawal, Daniel Fink, [Vargas and Kotyan, 2019] Danilo Vasconcellos Vargas and
Olivier Francon, Bala Raju, Hormoz Shahrzad, Arshak Shashank Kotyan. Robustness assessment for adversar-
Navruzyan, Nigel Duffy, et al. Evolving deep neural ial machine learning: Problems, solutions and a survey
networks. In Artificial Intelligence in the Age of Neural of current neural networks and defenses. arXiv preprint
Networks and Brain Computing, pages 293–312. Elsevier, arXiv:1906.06026, 2019.
2019.
[Vargas and Murata, 2017] Danilo Vasconcellos Vargas and
[Moosavi-Dezfooli et al., 2017] Seyed-Mohsen Moosavi- Junichi Murata. Spectrum-diverse neuroevolution with
Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal unified neural models. IEEE transactions on neural net-
Frossard. Universal adversarial perturbations. In 2017 works and learning systems, 28(8):1759–1773, 2017.
IEEE Conference on Computer Vision and Pattern
[Vargas and Su, 2019] Danilo Vasconcellos Vargas and Ji-
Recognition (CVPR), pages 86–94. Ieee, 2017.
awei Su. Understanding the one-pixel attack: Prop-
[Negrinho and Gordon, 2017] Renato Negrinho and Ge- agation maps and locality analysis. arXiv preprint
off Gordon. Deeparchitect: Automatically design- arXiv:1902.02947, 2019.
ing and training deep architectures. arXiv preprint
[Xu et al., 2017] Weilin Xu, David Evans, and Yanjun Qi.
arXiv:1704.08792, 2017.
Feature squeezing: Detecting adversarial examples in deep
[Papernot et al., 2016] Nicolas Papernot, Patrick McDaniel, neural networks. arXiv preprint arXiv:1704.01155, 2017.
Xi Wu, Somesh Jha, and Ananthram Swami. Distillation
[Yu and Kim, 2018] Dong-Pil Yu and Yong-Hyuk Kim. Is it
as a defense to adversarial perturbations against deep neu-
ral networks. In 2016 IEEE Symposium on Security and worth to approximate fitness by machine learning? inves-
Privacy (SP), pages 582–597. Ieee, 2016. tigation on the extensibility according to problem size. In
Proceedings of the Genetic and Evolutionary Computation
[Pham et al., 2018] Hieu Pham, Melody Guan, Barret Zoph, Conference Companion, pages 77–78, 2018.
Quoc Le, and Jeff Dean. Efficient neural architecture
[Zoph et al., 2018] Barret Zoph, Vijay Vasudevan, Jonathon
search via parameter sharing. In International Conference
on Machine Learning, pages 4092–4101, 2018. Shlens, and Quoc V Le. Learning transferable architec-
tures for scalable image recognition. In Proceedings of the
[Real et al., 2017] Esteban Real, Sherry Moore, Andrew IEEE conference on computer vision and pattern recogni-
Selle, Saurabh Saxena, Yutaka Leon Suematsu, Jie Tan, tion, pages 8697–8710, 2018.
Quoc V Le, and Alexey Kurakin. Large-scale evolution
of image classifiers. In Proceedings of the 34th Interna-
tional Conference on Machine Learning-Volume 70, pages
2902–2911. JMLR. org, 2017.
[Sabour et al., 2017] Sara Sabour, Nicholas Frosst, and Ge-
offrey E Hinton. Dynamic routing between capsules. In
Advances in neural information processing systems, pages
3856–3866, 2017.
[Sharif et al., 2016] Mahmood Sharif, Sruti Bhagavatula,
Lujo Bauer, and Michael K Reiter. Accessorize to a crime: