DeepXDE: A Deep Learning Library for Solving Differential Equations Lu Lu1 , Xuhui Meng1 , Zhiping Mao1 , George Em Karniadakis1 1 Division of Applied Mathematics, Brown University Providence, RI 02906 george karniadakis@brown.edu Abstract 1 Physics-informed neural networks Deep learning has achieved remarkable success in diverse We consider the PDE parameterized by λ for the solution applications; however, its use in solving partial differen- u(x) with x = (x1 , . . . , xd ) defined on a domain Ω ⊂ Rd : tial equations (PDEs) has emerged only recently. Here, we  ∂u ∂u ∂2u ∂2u  present an overview of physics-informed neural networks f x; ,..., ; ,..., ; . . . ; λ = 0, (PINNs), which embed a PDE into the loss of the neural ∂x1 ∂xd ∂x1 ∂x1 ∂x1 ∂xd network using automatic differentiation. PINNs solve inverse (1) problems similarly to forward problems. We also present a with suitable boundary conditions (BCs) B(u, x) = 0 on Python library for PINNs, DeepXDE. DeepXDE supports ∂Ω. For time-dependent problems, we consider time t as a complex-geometry domains based on the technique of con- special component of x. structive solid geometry, and enables the user code to be com- pact, resembling closely the mathematical formulation. We PDE(λ) introduce the usage of DeepXDE, and we also demonstrate ∂ the capability of PINNs and the user-friendliness of Deep- NN(x, t; θ) ∂t Tf ∂ û ∂ 2 û ∂t − λ ∂x2 XDE for two different examples. σ σ ∂2 ∂x2 x σ σ More recently, solving partial differential equations (PDEs) .. .. û Minimize t . . I û(x, t) − gD (x, t) Loss θ∗ via deep learning has emerged as a potentially new sub-field σ σ Tb under the name of Scientific Machine Learning. To solve a ∂ ∂n ∂ û ∂n (x, t) − gR (u, x, t) PDE via deep learning, a key step is to constrain the neu- BC & IC ral network to minimize the PDE residual, and several ap- proaches have been proposed to accomplish this. Compared to the traditional mesh-based methods, such as the finite dif- Figure 1: Schematic of a PINN for solving the diffusion ∂2u ference method and the finite e lement m ethod, d eep learn- equation ∂u ∂t = λ ∂x2 with mixed BCs u(x, t) = gD (x, t) ∂u ing could be a mesh-free approach by taking advantage of on ΓD ⊂ ∂Ω and ∂n (x, t) = gR (u, x, t) on ΓR ⊂ ∂Ω. the automatic differentiation, and could break the curse of dimensionality. Among these approaches, one could use the The algorithm of PINN is shown visually in the schematic PDE in strong form directly; in this form, automatic differ- of Fig. 1 solving a diffusion equation. We explain each entiation could be used directly to avoid truncation errors. step as follows. In a PINN, we first construct a neural net- This approach is called physics-informed neural networks work û(x; θ) as a surrogate of the solution u(x). Here, (PINNs). An attractive feature of PINNs is that it can be used θ = {W ` , b` }1≤`≤L is the set of all weight matrices and to solve inverse problems similarly to forward problems. bias vectors in the network û. One advantage of choosing In this paper, we present PINN algorithms implemented neural networks as the surrogate of u is that we can take the in a Python library DeepXDE (https://github.com/lululxvi/ derivatives of û with respect to x by the automatic differen- deepxde). DeepXDE can be used to solve multi-physics tiation. In the next step, we need to restrict û to satisfy the problems, and supports complex-geometry domains based PDE and BCs. We only restrict û on some scattered points, on the technique of constructive solid geometry (CSG), i.e., the training data T = {x1 , x2 , . . . , x|T | } of size |T |. T hence avoiding tedious and time-consuming computational is comprised of two sets Tf ⊂ Ω and Tb ⊂ ∂Ω, which are the geometry tasks. Last but not least, DeepXDE is designed to points in the domain and on the boundary, respectively. We make the user code stay compact and manageable, resem- refer Tf and Tb as the sets of “residual points”. To measure bling closely the mathematical formulation. the discrepancy between û and the constraints, we consider the loss defined as: Copyright c 2020, for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International L(θ; T ) = wf Lf (θ; Tf ) + wb Lb (θ; Tb ), (2) (CCBY 4.0).  2 3 Demonstration example  where Lf (θ; Tf ) = |T1f | x∈Tf f x; ∂x ∂ û P 1 ,...;λ , 2 We use PINNs and DeepXDE to solve inverse problems. A 1 Lb (θ; Tb ) = |Tb | x∈Tb kB(û, x)k22 , and wf and wb are the P diffusion-reaction system in porous media for the solute con- weights. In the last step, the procedure of searching for a centrations CA , CB and CC (A + 2B → C) is described by good θ by minimizing the loss L(θ; T ) using gradient-based ∂CA ∂ 2 CA 2 ∂CB ∂ 2 CB 2 optimizers is called “training”. ∂t = D ∂x2 − kf CA CB , ∂t = D ∂x2 − 2kf CA CB for x ∈ [0, 1], t ∈ [0, 10] with IC CA (x, 0) = CB (x, 0) = In inverse problems, there are some unknown parame- e−20x and BCs CA (0, t) = CB (0, t) = 1, CA (1, t) = ters λ in Eq. (1), but we have extra information on points CB (1, t) = 0. We estimate the diffusion coefficient D = Ti ⊂ Ω: I(u, x) = 0, for x ∈ Ti . PINNs solve inverse 2 × 10−3 and the reaction rate kf = 0.1 based on 40000 1 P by adding an2 extra term to Eq. (2): Li (θ, λ; Ti ) = problems observations of the concentrations CA and CB in the spatio- |Ti | x∈Ti kI(û, x)k2 . We then optimize θ and λ together: temporal domain. The identified D (1.98 × 10−3 ) and kf ∗ ∗ θ , λ = arg minθ,λ L(θ, λ; T ). (0.0971) are displayed in Fig. 3. A 16 B 3 2 DeepXDE usage Figure 3: The Parameter value Parameter value 12 2 identified values of In this section, we introduce the usage of DeepXDE. Deep- diffusion-reaction 8 True ρ Identified ρ 1 XDE makes the code stay compact and nice, True σ resembling Identified σ system converge closely the mathematical formulation. 4 Solving True β differential Identified β 0 True kf Identified kf True D Identified D to the true values equations in DeepXDE is no more than specifying the prob- during the training 0 -1 0 lem using the build-in modules, including 1 computational 2 3 4 do- 5 6 0 1 2 3 4 5 6 7 8 # Iterations (104) # Iterations (104) process. main (geometry and time), differential equations, ICs, BCs, constraints, training data, network architecture, and training hyperparameters. The workflow is shown in Procedure 1. Procedure 1 Usage of DeepXDE for solving differential equations. A B 1. Specify the computational domain using the geometry - module. A|B 2. Specify the differential equations using the grammar of TensorFlow. 3. Specify the boundary and initial conditions. A-B | & 4. Combine the geometry, PDE, and IC/BCs together into data.PDE or data.TimePDE for time-independent or A&B time-dependent problems, respectively. To specify train- ing data, we can either set the specific point locations, or Figure 2: CSG examples. (left) A and B represent the rectan- only set the number of points and then DeepXDE will gle and circle, respectively. A|B, A − B, and A&B are con- sample the required number of points on a grid or ran- structed from A and B. (right) A complex geometry (top) domly. is constructed from a polygon, a rectangle and two circles 5. Construct a neural network using the maps module. (bottom). 6. Define a Model by combining the PDE problem in Step 4 and the neural net in Step 5. In DeepXDE, The built-in primitive geometries in- 7. Call Model.compile to set the optimization hyperpa- clude interval, triangle, rectangle, polygon, rameters, such as optimizer and learning rate. The weights disk, cuboid and sphere. Other geometries can be in Eq. (2) can be set here by loss weights. constructed from these primitive geometries using three boolean operations: union (|), difference (-) and 8. Call Model.train to train the network from random intersection (&). This technique is called construc- initialization or a pre-trained model using the argument tive solid geometry (CSG), see Fig. 2 for examples. model restore path. It is extremely flexible to mon- DeepXDE supports four standard BCs, including itor and modify the training behavior using callbacks. Dirichlet, Neumann, Robin, and Periodic, and 9. Call Model.predict to predict the PDE solution at a more general BC can be defined using OperatorBC. different locations. The initial condition can be defined using IC. There are two networks available in DeepXDE: feed-forward neural network (maps.FNN) and residual neural network (maps.ResNet). It is also convenient to choose different References training hyperparameters, such as loss types, metrics, Lu, L.; Meng, X.; Mao, Z.; and Karniadakis, G. E. 2019. optimizers, learning rate schedules, initializations and Deepxde: A deep learning library for solving differential regularizations. equations. arXiv preprint arXiv:1907.04502.