Nonlocal models provide an improved predictive capability thanks to their ability to capture effects that classical partial differential equations fail to capture. Among these effects we have multiscale behavior and anomalous behavior such as super- and sub-diffusion. These models have become incredibly popular for a broad range of applications, including mechanics, subsurface flow, turbulence, plasma dynamics, heat conduction and image processing. However, their improved accuracy comes at a price of many modeling and numerical challenges. In this work we focus on the estimation of model parameters, often unknown, or subject to noise. In particular, we address the problem of model identification in presence of sparse measurements. Our approach to this inverse problem is based on the combination of 1. Machine Learning and Physical Principles and 2. a Unified Nonlocal Vector Calculus and Versatile Surrogates such as neural networks (NN). The outcome is a flexible tool that allows us to learn existing and new nonlocal operators. We refer to our technique as nPINNs (nonlocal Physics-Informed Neural Networks); here, we model the nonlocal solution with a NN and we solve an optimization problem where we minimize the residual of the nonlocal equation and the misfit with measured data. The result of the optimization are the weights and biases of the NN and the set of unknown model parameters.
Nonlocal equations are model descriptions for which the state of a system at any point depends on the state in a neighborhood of points, i.e. every point in a domain interacts with a neighborhood of points. As such, interactions can occur at
Sandia National Laboratories is a multimission laboratory managed and operated by National Technology and Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International, Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. This paper describes objective technical results and analysis. Any subjective views or opinions that might be expressed in the paper do not necessarily represent the views of the U.S. Department of Energy or the United States Government. Report 2019-14015.
Copyright c 2020, for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CCBY 4.0). distance, without contact. These models are such that they can capture effects that traditional PDEs fail to capture; in fact, their solutions can be irregular: non-differentiable, singular, and discontinuous. Among those effects, we mention: 1) Multiscale behaviors and discontinuities such as cracks and fractures and 2) Anomalous behaviors such as superand sub-diffusion. In case 1) we refer to nonlocal truncated operators where the neighborhood is a ball of radius (usually much smaller than the domain) surrounding any point. In case 2) refer to fractional operators where the interactions can be infinite ( = 1); a standard representative of this class is the fractional Laplacian operator ( )s.
As a consequence, nonlocal models provide an
improved predictive capability for several scientific and
engineering applications including fracture mechanics
In its simplest form, a nonlocal operator can be defined as Lu(x) = (u(y) u(x))k(x; y) dy; (1) Z
B (x) where B (x) is the ball of radius centered at x and where k is an application dependent kernel that determines the regularity properties of the solution. The integral form allows us to catch long-range forces and reduces the regularity requirements of the solution.
We consider nonlocal diffusion problems of the form (
Lu = f u = g x 2 x 2
I ; (2) where Rn is an open bounded domain and I is the interaction domain, a layer of thickness surrounding the domain where nonlocal boundary conditions must be prescribed for the well-posedness of the problem.
Two very important concerns arise when addressing the solution of (2).
Q1 Is (1) general enough? How broad is the class of nonlocal operators that can be described by one single formula and analyzed through one unified calculus? Q2 What is the “right” kernel for a given phenomenon? How can available data help determine the appropriate nonlocal model and its parameters? Can we design a unified datadriven tool for model identification and simulation of a broad class of nonlocal models? The first concern arises from the fact that in the literature we have independent definitions, formulations and theory of nonlocal models. Similarities are evident, but they have not been rigorously proved. This is addressed in the next section.
The purpose of a unified nonlocal notation and theory is to Connect the nonlocal and fractional communities that would benefit from each other’s research; Include as special cases the well-known classical differential calculus at the limit of vanishing interactions and the fractional calculus at the limit of infinite interactions; Provide the groundwork for new model discovery thanks to the broad class of operators that it describes; Describe intrinsically nonlocal phenomena that have not been analyzed or used due to the lack of theory.
Guide algorithm/discretization/solver design.
In this work we introduce a generalized nonlocal operator, in the spirit of a unified calculus, that bridges local, truncated nonlocal and fractional diffusion operators:
L ;su(x) = C ;s
Z
u(x) B (x) jx
u(y) yjn+2s dy (3) where Cs; is such that the corresponding solutions span a broad range of nonlocal diffusion processes including local and fractional diffusion at the limit of vanishing and increasing nonlocality, i.e.
lim L ;su = !0 u and lim !1
L ;su = ( )su.
A unified computational framework The unified nonlocal vector calculus, and more specifically the operator in (3) provides us with a universal definition of parametrized nonlocal operators that describe both wellknown nonlocal phenomena and may describe new intrinsically nonlocal phenomena not yet analyzed and used due to lack of theory. However, the universal nature of these new mathematical models and the abundance of data raise important questions.
Q3 What are the true model parameters and s? Q4 How can we deal with data sparsity and noise (the forcing term f and the nonlocal boundary condition g in (2) may be sparse or subject to noise)? We propose a new approach to model learning that is in stark contrast with previously developed UQ and PDEconstrained-like optimization techniques. The game changer is the combination of 1) Machine Learning and Physical Principles, and 2) Unified Calculus and Versatile Surrogates, such as neural networks. The outcome is a DataDriven Physics-Informed tool for learning new complex nonlocal phenomena.
We refer to our strategy as nPINNs (nonlocal
PhysicsInformed Neural Networks); this is an extension of PINNs
1 um; i;ns Loss(u; ; s) = 2 2 xi2Tf X (L ;suNN (xi)
fm(xi))2+ X (uNN (xj)
um(xj))2, xi2Tu where the minimization with respect to u must be regarded as minimization with respect to the weights and biases of the NN. The two, distinct, training sets in 1 depend only on data availability and are not necessarily associated with quadrature points. Note that Loss has a physics-driven and a data-driven component: the first term controls the residual of the nonlocal equation, whereas the second the mismatch between solution and data. The outcome of the optimization are the weights and biases of the NN and the model parameters. This strategy
Is as accurate as any other discretization method for the forward problem. As an example, numerical tests show that it has the same convergence rate, as the number of training points increases, of fPINNs and of a standard Finite Difference discretization. However, due to the increased computation cost, nPINNs is not yet recommended for the solution of forward problems.
Requires minimal implementation effort: available solvers can be used as black boxes.
We tested this method on one-dimensional forward and inverse problem (to illustrate our theoretical findings and learn model parameters) and on two- and three- dimensional forward problems (to show applicability in higher dimensions). Also, we applied nPINNs to the solution of turbulent Couette flow for the estimation of the dispersion rate s and the characteristic length . Computational results are promising and show that the versatility of NN allows one to describe complex phenomena, to identify model parameters and to handle data sparsity.
One-dimensional example In Figure 1 we report the outcome of our algorithm (steps 1–3) for the estimation of and s. For =(0; 1) and [ I =( ; 1 + ), we consider the nonlocal diffusion problem (2) with g=0, f =sin(2 x) and L defined as in (3). The training data um are generated via accurate solution of (2) with parameters ( ; s )=(14; 0:8); we refer to these values as true values and represent them with a yellow star in the plot. The training points are 100 uniformly spaced points in [ I . We run the algorithm for two initial guesses, represented by the blue dots and report their trajectories. Both of them, see pink dots in both plots, converge to the true values. The optimal uNN corresponding to the estimated parameters are accurate for both initial guesses; in fact, their relative errors are of the order of 10 4.
Silling, S. 2000. Reformulation of elasticity theory for discontinuities and long-range forces. Journal of the Mechanics and Physics of Solids 48:175–209.