Adversarial Variational Optimization of
             Non-Differentiable Simulators

               Gilles Louppe1 , Joeri Hermans1 , and Kyle Cranmer2
 1
     Department of Electrical Engineering and Computer Science, University of Liège
                   2
                     Department of Physics, New York University



        Abstract. Complex computer simulators are increasingly used across
        fields of science as generative models tying parameters of an underly-
        ing theory to experimental observations. Inference in this setup is of-
        ten difficult, as simulators rarely admit a tractable density or likelihood
        function. We introduce Adversarial Variational Optimization (AVO), a
        likelihood-free inference algorithm for fitting a non-differentiable gen-
        erative model incorporating ideas from generative adversarial networks,
        variational optimization and empirical Bayes. We adapt the training pro-
        cedure of generative adversarial networks by replacing the differentiable
        generative network with a domain-specific simulator. We solve the result-
        ing non-differentiable minimax problem by minimizing variational upper
        bounds of the two adversarial objectives. Effectively, the procedure re-
        sults in learning a proposal distribution over simulator parameters, such
        that the JS divergence between the marginal distribution of the syn-
        thetic data and the empirical distribution of observed data is minimized.
        We evaluate and compare the method with simulators producing both
        discrete and continuous data.

        Keywords: Likelihood-free Inference · Simulator-based inference · Physics


Adversarial Variational Optimization (AVO). In this work, we introduce Ad-
versarial Variational Optimization [1], a likelihood-free inference algorithm for
non-differentiable, implicit generative models. We adapt the adversarial train-
ing procedure of generative adversarial networks (GANs) by replacing the im-
plicit generative network with a domain-based scientific simulator, and solve the
resulting non-differentiable minimax problem by minimizing variational upper
bounds of the adversarial objectives. The objective of the algorithm is to match
the marginal distribution of the synthetic data to the empirical distribution of
observations.

Method. The alternating stochastic gradient descent on the discriminator and
generator losses Ld and Lg in GANs implicitly assumes that the generator g is a
differentiable function. In the setting where we are interested in estimating the
parameters of a fixed non-differentiable simulator, gradients either do not exist
or are not accessible. As a result, gradients ∇θ Lg cannot be constructed and the
optimization procedure cannot be carried out.


Copyright c 2019 for this paper by its authors. Use permitted under Creative
               Commons License Attribution 4.0 International (CC BY 4.0).
2       G. Louppe, J. Hermans and K. Cranmer

   In this work, we rely on variational optimization to minimize Ld and Lg ,
thereby bypassing the non-differentiability of g. We consider a proposal distri-
bution q(θ|ψ) over the parameters of the simulator g and alternately minimize
the variational upper bounds
                             Ud (φ) = Eθ∼q(θ|ψ) [Ld (φ)]                            (1)
                             Ug (ψ) = Eθ∼q(θ|ψ) [Lg (θ)]                            (2)
respectively over φ and ψ. The discriminator d is therefore no longer pit against a
single generator g, but instead against a hierarchical family of generators induced
by the proposal distribution.
    When updating the discriminator parameters φ, unbiased estimates of ∇φ Ud
can be obtained by directly evaluating the gradient of Ud over mini-batches of
real and synthetic data. When updating the proposal parameters ψ, ∇ψ Ug can
be estimated as described in the previous section with f (θ) = Lg (θ). That is,
               ∇ψ Ug = Eθ∼q(θ|ψ), [∇ψ log q(θ|ψ) log(1 − d(x̃; φ))],                (3)
                           x̃∼p(x|θ)

which we can approximate with mini-batches of synthetic data.

Results. Figure 1 summarizes our results for AVO, ABC-SMC and BOLFI on
four inference problems. The figure clearly indicates AVO works better on av-
erage compared to ABC-SMC and BOLFI. We attribute this superior perfor-
mance primarily to the fact that AVO is not limited by the deficiencies of a
hand-crafted summary statistic. Instead, AVO benefits from a high-capacity dis-
criminator that dynamically adapts to the inference problem and to the current
proposal.




           Fig. 1: Benchmark results comparing AVO against ABC-SMC and BOLFI.




References
1. Louppe, G., Hermans, J., Cranmer, K.: Adversarial variational optimization of non-
   differentiable simulators. In: The 22nd International Conference on Artificial Intel-
   ligence and Statistics. pp. 1438–1447 (2019)