Introduction

One area where active learning plays a really important role is black-box optimization (BBO), i.e., optimization of objective functions for which no analytical description is provided. It employs optimization methods that need as input only points in the search space paired with respective values of the objective function obtained in a non-analytical way, e.g. from sensors, in experiments or through numerical simulations. Most frequently used are evolutionary optimization approaches, such as evolution strategies, genetic algorithms, and differential evolution, or other metaheuristics, such as particle swarm optimization.

Because BBO methods receive only information about values of the objective function, they typically need many such values. This is a problem in situations when evaluating the black-box objective function is time-consuming and/or expensive. That is frequently the case if it is evaluated empirically in experiments. For example, for the evolutionary optimization tasks described in the book [1], the evaluation of a comparatively small generation of a genetic algorithm can sometimes take more than a week and cost more than 10000 e. To deal with expensive evaluations, continuous BBO has in the late 1990s and early 2000s adopted an approach called surrogate modelling or metamodelling [2,3,4,5,6,7,8]. In principle, a surrogate model is any regression model that with a sufficient fidelity approximates the original black-box objective function, restricting the necessity of its evaluation only to a small proportion of points, whereas everywehere else, only the surrogate model is used.

Selecting the points in which the original objective function should be evaluated is a step in which active learning is involved. However, it is not active learning of a regression model although the surrogate model itself is a regression model. The reason is that its utility functions are not based on the model, like are the commoly used utility functions uncertainty decrease, model performance, diversity, or surprise-novelty. Instead, they are based on the BBO, the most common being minimizing the objective function for a given evaluation budget, and minimizing the evaluation budget for a given objective-function threshold. Nevertheless, even active learning in surrogate-assisted BBO follows the basic priciple of active learning: to actively select next model inputs according to the considered utility function.

The earliest kinds of surrogate models in continuous BBO were low-order polynomials and artificial neural networks (ANNs) of the kind multilayer perceptron (MLP). The former have always remained a suitable choice in situations when enough evaluations of the original black-box objective function are affordable for the approximation properties of polynomials to be in effect. On the other hand, surrogate modelling for substantially less evaluations of the original objective has during the last two decades undergone further development. MLPs were soon replaced with another kind of ANNs, radial basis function networks (RBFs), which better fit local peculiarities of an objective function landscape. Those networks, however, have since the late 2000s been superseded by other kinds of surrogate models, primarily Gaussian processes (GPs), but also ranking support vector machines (RSVMs), and random forests (RFs). GPs are currently the most successful kind of surrogate models for BBO with small evaluation budget of functions with complicated multimodal landscapes, mainly due to their ability to assess the uncertainty of the estimate of the original objective function in a given point, more precisely, to provide the probability distribution of this estimate. That property of GPs allows to combine the original BBO method, e.g. an evolutionary one, with Bayesian optimization.

Consequently, only little attention has been paid to ANN-based surrogate models in continuous BBO during the last 15 years. This contrasts with the intense and successful development of the ANN area during that time, which suggests that ANNs again have the potential to serve as promising surrogate models. This paper attempts to bring a small contribution to research into that potential, presenting in addition a review of possible directions for such a research, connected with different classes of neural networks. Moreover, it also points out that ANNs can serve as the basis for transfer learning between surrogate-assisted BBO of different functions.

The next section surveys important aspects and key methods concerning surrogate-assisted continuous BBO. The review of possible research directions concerning the usability of modern neural networks in surrogate-assisted BBO is presented in Section 3. Finally, Section 4 reports an experimental contribution to one of those research directions.

Surrogate-Assisted Continuous BBO

Surrogate modelling for continuous BBO relies on combination and interaction of three components: a regression model serving as a surrogate of the original black-box objective function, a BBO method seeking the optimum of that objective function, and a strategy when to evaluate the original objective function and when its surrogate model. That strategy is in the context of evolutionary BBO usually called evolution control [9,10,11,12,13]. There are two other aspects, namely observing constraints on the feasible set of the black-box objective function (cf. e.g. [14,15]), and generalizing surrogate modelling from single objective to multiple objectives (cf. e.g. [16,17]), however, we will restrict our attention to single-objective unconstrained optimization.

As already mentioned in the introduction, the regression models that are the most suitable kind of surrogate models if sufficiently many evaluations of the original black-box objective function are affordable, are low-order polynomials, typically quadratic functions [18,19,20,21,22]. For substantially less evaluations, the most traditional kind have been MLPs [23,9], soon replaced with RBFs [24,25,26,21,22], and since the late 2000s with GPs a.k.a. kriging [27,28,11,29,30]. Occasionally, RBFs were used as local models in combination with GP-based global models [31]. Other kinds of surrogate models employed during the last decade include decision trees [32], RFs [33,34,32], and RSVMs [35,36]. The last one has an exceptional property of invariance with respect to order-preserving transformations of the objective function. This is important in situations when the BBO algorithm possesses such invariance, a frequently encountered property of evolutionary algorithms. On the other hand, the surrogate modelling methods proposed in [11] and [28] use GPs to perform preselection based on a partial ordering that is also invariant with respect to order-preserving transformations. More importantly, the adaptive function value warping approach recently proposed in [37] aims at providing such invariance to any surrogate model. As a final remark to different kinds of surrogate models, important works about that topic always consider several kinds [38,12,39,20,32], to compare them and select the best among them, and in [22,39] also to aggregate their results, thus providing a team of surrogate models.

As to the BBO methods, not only the two most important kinds of surrogate models, i.e. low-order polynomials [18,19,20], and GPs [26,28,11,29,30], but also the less common RBFs, RFs, and RSVMs [24,36,33,34] are most often combined with the Covariance matrix adaptation evolution strategy (CMA-ES). That is not surprising because CMA-ES has already in the 2000s become a state-of-the-art approach to single-objective unconstrained continuous BBO. Basically, the CMA-ES evolves a Gaussian estimate of the position of the minimum of the original objective function. That evolution relies on simultaneous adaptation of the vector mean of the Gaussian estimate, of the scalar step size, and of the covariance matrix. For more details of this sophisticated evolution strategy, the reader is referred to the journal papers [40,41]. GPs were also combined with other evolutionary optimization methods [27,42], and GPs, polynomials, and RBFs were combined with particle swarm optimization [22] and with memetic optimization [25]. Moreover, GPs are used in black-box optimization in two different ways. In connection with evolutionary and similar BBO methods, they serve as a regression model evaluated instead of the original objective function. In addition, they also play a key role in Bayesian optimization, which then relies on GP-estimates of probability distributions of values of the original objective. Those probability distributions enable several ways of searching for optima of that objective function, each of them governed by a specific assessment of uncertainty of the objective function estimate, commonly called acquisition function [43,44,45]. Occasionally, Bayesian optimization is combined with CMA-ES. For example in [46], optimization switches from the most traditional Bayesian optimization method, EGO (Efficient Global Optimization) [43], to CMA-ES.

Finally, evolution control has been since the first surrogate-assisted BBO methods performed basically in two ways, generation-based, and individual-based. In the generation based, all points are in some generations evaluated with the true objective function, and in the remaining generations with the model. On the other hand, in every generation of the individual-based evolution control, based on the evaluation of all points with the model, a preselection of points to be evaluated with the true objective function is performed [9]. In most of the surrogate-assisted methods, however, the evolution control is specifically tailored to the respective method. Noteworthy, the authors of [13] investigated mutually replacing the evolution control of two important polynomial-assisted methods lmm-CMA [18,19] and lq-CMA-ES [20], and of two variants of the GP-assisted method DTS-CMA-ES [47,12] with the evolution control of the others. According to their findings, the success of those important methods is definitely not limited to using the respective specific tailored evolution control. The surrogate-assisted black-box optimization methods constructing several surrogate models simultaneously either aggregate them to a team [25,22] or complement the evolution control by a classifier selecting the most appropriate among those models. Important examples of classifiers used in this context are ANNs [48,49,50], and classification trees [51,52]. Their learning can be viewed as metalearning because it is based on metafeatures, i.e. properties empirically characterizing the objective function landscape and the BBO method [21,32,49,53]. Apart from classification according to the appropriateness of the surrogate model for the considered data, metalearning can be also used for regression of model error on the combination of values of metafeatures [54].

Usability of Modern Neural Networks in Surrogate-Assisted BBO

This section primarily reviews eight kinds of modern neural networks that we consider worth a research into their ability to serve as surrogate models in BBO. A high-level overview of those kinds of ANNs is given in Table 1, which for each of them mentions whether such research has already started. In Subsection 3.1, two kinds integrating GPs into ANNs are recalled. Subsection 3.2 recalls three kinds of ANNs providing the most advantageous property of GPs, their ability to estimate the distribution of black-box objective function values. Finally, in Subsection 3.3, three well-known kinds of modern neural

ANNs

Research into its ability + main references to serve as surrogate model in BBO MLPs with a GP as the final layer [55,56] First investigations [57,58] Deep GP networks [59,60,61,62,63] Not Tangent kernel networks [64,65] Not Prior networks [66,67,68,69,70] First investigations [71] Ensembles of neural networks [72,73,74,75,76] First investigations [this paper] Variational autoencoders [77,78] Not Generative adversarial networks [79,80] Not Transformers [81,82] Not networks, namely variational autoencoders, transformers, and generative adversarial networks, are recalled due to the fact that they have already proven useful in the related area of Bayesian optimization. In addition, Subsection 3.4 is devoted to knowledge transfer in surrogate-assisted BBO, which relates to the usability of modern neural networks through their important role in transfer learning.

Integration of GPs into ANNs

The integration of GPs into ANNs has been proposed on two different levels:

1. At the layer level -a GP serves as the final layer of an MLP [55,56]. Integration on that level is based on the following two assumptions: (i) If 𝑛 𝐼 denotes the number of the ANN input neurons, then the ANN computes a mapping net of 𝑛 𝐼 -dimensional input values into the set 𝒳 on which is the GP defined. Consequently, the number 𝑛 𝑂 of neurons in the last hidden layer fulfills 𝒳 ⊂ R 𝑛 𝑂 , and the ANN maps an input 𝑣 into a point 𝑥 = net(𝑣) ∈ 𝒳 , corresponding to an observation 𝑓 (𝑥 + 𝜀) governed by the GP, where 𝜀 is a zero-mean Gaussian noise. From the point of view of the ANN inputs, the GP is now 𝒢𝒫(𝑚 GP (net(•)), 𝜅(net(•), net(•))), where 𝑚 GP is the mean function, and 𝜅 is the covariance function of the GP [83]. (ii) The GP mean 𝜇 is assumed to be a known constant, thus not contributing to the GP hyperparameters, and independent of net 2. At the level of individual neurons -GPs can replace all hidden and output neurons of an MLP. This kind of neural networks is commonly called deep Gaussian process [59,60,61,62,63,84,85,86,87,88,89,90].

Integration on both levels has been developed primarily for Bayesian modelling and optimization. Nevertheless, GPs integrated as the last layer of MLPs have been used as surrogate models in a CMA-ES-driven BBO [57,58]. In particular, those surrogate models incorporate GPs with five commonly employed covariance functions linear, quadratic, rational quadratic, squared exponential, and Matérn 5 2 , as well as with one composite covariance function superposing the quadratic and squared exponential. Those 6 models were compared in [57] from the point of view of regression accuracy, evaluated on a large dataset collected during many previous runs of DTS-CMA-ES on the collection of 24 noiseless benchmarks from the Comparing Continuous Optimizers platform [91,92] (cf. Section 4) in dimensions 2, 3, 5, 10, and 20. Then in [58], they were compared on the same benchmarks in the same dimensions from the point of view of the success of surrogate-assisted optimization with CMA-ES. Unfortunately, neither of those comparisons included more traditional surrogate models nor the CMA-ES without surrogate assistance. To our knowledge, the only comparison that included both a GP integrated as the last layer of an MLP, and more traditional surrogate models, was the comparison from the point of view of regression accuracy in [93]. However, it included only one such integrated surrogate model, with the GP using the most simple covariance function -the linear one, in addition to the traditional GP-based surrogate models with eight different covariance functions, including the five listed above.

ANNs Estimating the Distribution of Black-Box Objective Function Values

In our opinion, the property of GPs most advantageous from the point of view of surrogate modelling is that they estimate the whole distribution of a predicted value of the original black-box objective function. Recall from Section 2 that due to that property also ensembles of regression trees -RFs -are used as surrogate models [33,34,32]. This draws attention to those modern neural networks that also allow estimation of such a distribution. Basically, there are three classes of them, differing in the way how that estimate can be obtained.

1. The multivariate normal distribution underlying GPs is actually the asymptotic distribution for network width increasing to infinity. Such results have been established for several kinds of ANNs [94,88,95,96,97]. In addition, closely related is the infinite width limit of the neural tangent kernel, which governs the kernel gradient of the functional cost used in MLP regression [64,65].

Although those results have great theoretical value, there can be a serious disparity between the infinite width results and their finite width counterparts [77]. Therefore, it is unclear whether they can be applied to surrogate modelling. 2. The distribution of a predicted value, or more precisely the parameters of such a distribution, can be directly learned by an ANN. The best-known kind of such neural networks are the prior networks, learning the parameters of a normal-inverse Wishart distribution, which is the conjugate prior to a multivariate normal distribution [66,67,68,69,70,98]. Prior networks belong to a broader class of evidential neural networks [99,100,101,102,103]. Their name refers to the fact that they follow the basic principle of the Dempster-Shafer theory of evidence [104] -to fall back onto prior belief for unfamiliar data. 3. An estimate of the distribution of a predicted value is produced by an ensemble of neural networks.

Important kinds of such ensembles are ensembles obtained through diversification of training data [105,106], ensembles obtained through diversification of network properties [107,108,109], a specific subgroup of which are ensembles in which the diversification is achieved through diverse activation functions [76], ensembles obtained through negative correlation learning [110,111,112], bagging ensembles [72,113], boosting ensembles [114,115] deep ensembles [73,74,116] including deep echo-state network ensembles [117], and anchored ensembles [75] with a later modification random activation function (RAF) ensembles [76]. RAF ensembles take over the principle of anchored ensembles that regularization is performed not with respect to zero, but with respect to the initialization values of the parameters, which are assumed normally distributed. Differently to an anchored ensemble, however, an RAF ensemble uses varied ativation fuctions from an a priori specified set of size 𝑛 AF . From that set, the activation function is chosen randomly, apart from the first 𝑛 AF members of the ensemble, among which each activation function occurs exactly once. We consider this last mentioned kind of ensembles as the state of the art.

To our knowledge, the only ANNs estimating the distribution of function values that have already been used as surrogate models in BBO, are prior networks. In [71], the prediction accuracy of four versions has been evaluated on the above mentioned dataset from previous runs of DTS-CMA-ES. This direction of research is continued by the present paper: Section 4 reports results for CMA-ES surrogate-assisted by two variants of RAF ensembles.

ANNs Found Useful in Bayesian Optimization

Recall from Section 2 that GPs, simultaneously with their importance as surrogate models in BBO with non-Bayesian methods, such as CMA-ES, also play a crucial role in Bayesian optimization. That is why this subsection lists three well-known kinds of modern neural networks that have been recently found useful in Bayesian optimization. In our opinion, this indicates that they are worth investigating whether they could be used also in surrogate-assisted BBO.

1. Variational autoencoders have been utilized in Bayesian optimization because they allow for optimization in a lower-dimensional latent space [77,78]. 2. The generative adversarial networks (GANs) paradigm has been recently shown to be applicable to BBO: A generator proposes samples that align with the distribution of low values (or even the optimal value) of the black-box function, while one or more discriminators classify samples based on whether they belong to that distribution [79,80]. 3. Transformers have proven effective in estimating complex prior distributions for Bayesian optimization [81,82]. Notably, an OptFormer transformer trained on Google Vizier [118], the largest hyperparameter optimization (HPO) database, achieved superior HPO outcomes compared to GP-based Bayesian optimization [81]. Furthermore, the recently introduced transformer-based Prior-data Fitted Networks [82] can mimic Gaussian Processes (GPs) and Bayesian networks, while also incorporating additional information into the prior.

ANN-Based Transfer Learning for Surrogate-Assisted Black-Box Optimization

Obtaining accurate surrogate models in the initial stages of BBO is challenging due to the scarcity of data points with evaluated objective function values. That can be mitigated by leveraging knowledgetransfer learning. And a connection of modern kinds of neural networks with transfer learning is even more obvious than with active learning. Indeed, transfer learning is nowadays one of the areas where ANNs play most important role [119,120,121]. Different types of ANNs have been utilized to this end, including convolutional [122,123], recurrent [124], autoencoder [125,126], GAN [127,128,129], and transformer [81]. In the context of the research direction pursued in this paper, most interesting are those that also have connections to BBO: (i) Four ANN-based transfer learning approaches draw inspiration from the GAN paradigm. CoGAN trains two GANs to generate the source and target, respectively, achieves a domain invariant feature space by tying the high layers parameters of the two GANs, and performs domain adaptation by training a classifier on the discriminator output [130]. Adversarial discriminative domain adaptation learns first a discriminative representation using the labels in the source domain, and then, using a domain-adversarial loss, a separate encoding that maps the target data to the same space through an asymmetric mapping [127]. Minimax-game-based selective transfer learning employs a selector and a discriminator to identify source domain data resembling the target domain's distribution, and distinguish genuine target domain data from selected source domain data, respectively [129]. Selective adversarial network addresses negative transfer by excluding outlier classes from the source domain selection, and maximizing the similarity between source and target domain data distributions [128]. (ii) An autoencoder for transfer learning, described in [125,126], incorporates embedding and label encoding layers. The embedding layer reduces the disparity between instance distributions from the source and target domains, while the label encoding layer utilizes a softmax regression model to encode label information from the source domain.

(iii)

The transformer OptFormer has demonstrated competitiveness with specific transfer learning methods, although its usage leans more toward metalearning than traditional transfer learning [81].

Experimental Evaluation of RAF Ensembles

This section describes a small experimental contribution to one of the above surveyed possible research directions: RAF ensembles are experimentally evaluated as surrogate models for CMA-ES. The experiments were performed on the probably most commonly used platform for experimenting in continuous optimization -COCO ( Comparing Continuous Optimizers) [92]. COCO contains severeal suites of benchmark functions, our evaluation was performed with the most traditional suite, which is the bbob suite [92]. It consists of 24 dimension-scalable noiseless benchmark functions, the definitions of which have been given in [91]. Each function is used in 15 differently rotated and/or translated instances. The employed benchmarks forming the bobo suite are surveyed in Appendix A.

Considered Variants of RAF Ensembles

As activation functions forming an RAF ensemble, we employed those included in the implementation [131], to which the RAF paper refers [76]. They are listed in Appendix B. We used them in two variants of RAF ensembles:

1. An RAF ensemble of size 5 trained directly using the above mentioned implementation [131], and aggregated by the empirical mean. In the results, it will be denoted simply RAF. 2. An ensemble of size 5, in which the differences of values of the original black-box objective function with respect to its median are first transformed to their logarithms before using [131] in the logarithmic scale to train the ensemble. This transformation attempts to deal with situations when the function returns in many points values close to the median. The aggregation function is again the empirical mean, which in terms of the data before the logarithmic transformation actually corresponds to the empirical geometric mean. That version will be in the results denoted RAF-log.

Considered CMA-ES Variants for Comparison

CMA-ES surrogate-assisted by the above mentioned two variants of RAF ensembles was compared with CMA-ES without surrogate modelling, as well as with two earlier surrogate-assisted variants of CMA-ES:

3. CMA-ES without surrogate modelling was used in an implementation that is in the COCO data archive [132] called default-CMA-ES, and described as "default CMA-ES from the pycma module, version 3.3.0". Here, it will be in the results denoted simply default. 4. DTS-CMA-ES [12], using a surrogate GP with the covariance function Matérn 5 2 . In the results, it will be denoted simply DTS. [20], which will be in the results denoted simply lq.

lq-CMA-ES

Evolution Control

Whereas DTS-CMA-ES and lq-CMA-ES have each their own evolution control, for the two variants of RAF ensembles was necessary to propose when to evaluate a given point 𝑥 by the original black-box objective function 𝐹 bb , and when by its surrogate model 𝐹 sm . We decided to use a modification of the lq-CMA-ES evolution control. That modification is described below in Algorithm 1 using the notation 𝜏 ((𝑦 1 , . . . , 𝑦 𝑘 ), (𝑧 1 , . . . , 𝑧 𝑘 )) for the Kendall correlation coefficient between the sequences (𝑦 1 , . . . , 𝑦 𝑘 ) and (𝑧 1 , . . . , 𝑧 𝑘 ), and the notation 𝜌 for the ranking function on R 𝑑 , i.e., 𝜌 : R 𝑑 → Π(𝑑) with Π(𝑑) denoting the set of permutations of {1, . . . , 𝑑}

such that ∀𝑦 ∈ R 𝑑 : (𝜌(𝑦)) 𝑖 < (𝜌(𝑦)) 𝑗 ⇒ 𝑦 𝑖 ≤ 𝑦 𝑗 . (1)

Results

In Tables 2-3, the two considered variants of RAF ensembles, and three considered other CMA-ES variants, are compared based on the difference between the optimal value of the objective function, and its value achieved for a given evaluation budget. The achieved values were averaged over the 15 instances provided by the COCO benchmark suite in each dimension for each of the 24 noiseless functions listed in Appendix A. The comparisons were performed separately for each of the five above described groups of those functions, and subsequently also for all 24 of them, each time including the instances in dimensions 2, 3, 5, 10, and 20. For each evaluation budget, hence, six evaluations were

Table 2

Comparison of CMA-ES surrogate-assisted by RAF, and by RAF-log, with CMA-ES without surrogate modelling, with lq-CMA-ES, and with DTS-CMA-ES, for evaluation budget 3×dimension. Each cell of each sub-table records the number of function-dimension combinations, for which the method in the row achieved with the evaluation budget a lower value, averaged over the 15 COCO instances, than the method in the column. Ties within the considered precision are halved between both methods. If the Friedman test rejected the hypothesis of equivalence of all methods, and according to the subsequent Wilcoxon signed-rank test with Holm correction, the method in the row is significantly better than the method in the column, the number in the cell is in bold with * for the familywise level 5 %, and with ** for the familywise level 1 %.

Table 3

Comparison of CMA-ES surrogate-assisted by RAF, and by RAF-log, with CMA-ES without surrogate modelling, with lq-CMA-ES, and with DTS-CMA-ES, for evaluation budget 50×dimension. Each cell of each sub-table records the number of function-dimension combinations, for which the method in the row achieved with the evaluation budget a lower value, averaged over the 15 COCO instances, than the method in the column. Ties within the considered precision are halved between both methods. If the Friedman test rejected the hypothesis of equivalence of all methods, and according to the subsequent Wilcoxon signed-rank test with Holm correction, the method in the row is significantly better than the method in the column, the number in the cell is in bold with * for the familywise level 5 %, and with ** for the familywise level 1 %. 2 were conducted for the evaluation budget 3×dimension, while the comparisons in Table 3 were conducted for the evaluation budget 50×dimension.

The results of each of those 12 comparisons were subsequently assessed for statistical significance. First, the hypothesis that all five considered methods are equivalent was tested by the Friedman test. With the exception of both comparisons for multi-modal functions with adequate global structure, the test rejected that hypothesis on the familywise significance level 5%, using the Holm procedure for multiple-hypothesis correction [133]. This rejection justified testing the equivalence of any two among the five methods. We adopted the arguments of [134] that, in machine learning, the Wilcoxon signed-rank test is more appropriate for this purpose than the post-hoc tests presented in [135] and [133]. If for particular two methods, the Wilcoxon signed-rank test rejected the hypothesis that they are equivalent, then in the respective table, their comparison in the row corresponding to the method that was more frequently better is shown in bold italics.

The results in Tables 2-3 primarily confirm the superior performance of the methods lq-CMA-ES, and DTS-CM-ES. In the two comparisons based on all 120 noiseless benchmark functions, each of them is for both considered budgets significantly better not only than default CMA-ES, but also than CMA-ES surrogate-assisted by the two variants of RAF ensembles. Moreover, lq-CMA-ES is also among the 10 comparisons based on individual groups of functions 6 times significantly better than default CMA-ES, and 7 times, respectively 5 times significantly better than CMA-ES surrogate-assisted by RAF, respectively by RAF-log. For DTS-CMA-ES, the results of the 10 comparisons based on individual groups of functions are less convincing: 3 times significantly better than default CMA-ES, 3 times than CMA-ES surrogate-assisted by RAF, and only once than CMA-ES assisted by RAF-log. As to a comparison between the two variants of RAF ensembles, the differences among them were not significant apart from unimodal functions with high conditioning, for which CMA-ES achieves significantly better results if assisted by RAF than if assisted by RAF-log.

The different progress of optimization performed by each of the compared methods is illustrated, always in three particular dimensions, by means of optimization-progress plots. They show the average difference Δ 𝑓 between the optimal and achieved value of the objective function over the 15 COCO instances. For that illustration, we have chosen the functions 𝑓 9 (Figure 1), 𝑓 18 (Figure 2), and 𝑓 20 (Figure 3). We can see that optimisation using CMA-ES surrogate-assisted by RAF or RAF-log sometimes leads to similarly fast decrease of the objective function as, or even faster than, optimization using the state-of-the-art methods DTS-CMA-ES or lq-CMA-ES. In Figure 1, this is the case for RAF-log in dimension 2. In Figure 2, dimesnion

Conclusion

The paper was motivated by our opinion that the intense and successful development of artificial neural networks during the last 15 years suggests that they again have the potential to be important for active learning in surrogate-assisted BBO. It surveyed possible directions of research into that potential, including closely connected research into neural-network-based transfer learning for surrogate modelling. Moreover, it recalled the first published investigations in some of those directions, and added a new contribution to the emerging mosaic of those investigations. The fact that the main purpose of the experimental section of the paper is to contribute to the mosaic of emerging investigations should be epmhasized especially in context of the obtained experimental results. It justifiess that there is no significant difference between using CMA-ES surrogate-assisted by RAF ensembles and using it alone, as well as that results with RAF-ensemble-based surrogate models are significantly worse than results with the state-of-the-art surrogate-assisted CMA-ES variants, lq-CMA-ES, and DTS-CMA-ES. This is an obvious limitation not only of RAF ensembles, but of all above surveyed kinds of neural networks that have been so far investigated as surrogate models for CMA-ES. On the other hand, as the survey has shown, there are many more other possibilities for such investigations within future research.