=Paper=
{{Paper
|id=Vol-2587/article_14
|storemode=property
|title=DeepXDE: A Deep Learning Library for Solving Differential Equations
|pdfUrl=https://ceur-ws.org/Vol-2587/article_14.pdf
|volume=Vol-2587
|authors=Lu Lu,Xuhui Meng,Zhiping Mao,George Em Karniadakis
|dblpUrl=https://dblp.org/rec/conf/aaaiss/0010MMK20
}}
==DeepXDE: A Deep Learning Library for Solving Differential Equations==
https://ceur-ws.org/Vol-2587/article_14.pdf
DeepXDE: A Deep Learning Library for Solving Differential Equations
Lu Lu1 , Xuhui Meng1 , Zhiping Mao1 , George Em Karniadakis1
1
Division of Applied Mathematics, Brown University
Providence, RI 02906
george karniadakis@brown.edu
Abstract 1 Physics-informed neural networks
Deep learning has achieved remarkable success in diverse We consider the PDE parameterized by λ for the solution
applications; however, its use in solving partial differen- u(x) with x = (x1 , . . . , xd ) defined on a domain Ω ⊂ Rd :
tial equations (PDEs) has emerged only recently. Here, we
∂u ∂u ∂2u ∂2u
present an overview of physics-informed neural networks f x; ,..., ; ,..., ; . . . ; λ = 0,
(PINNs), which embed a PDE into the loss of the neural ∂x1 ∂xd ∂x1 ∂x1 ∂x1 ∂xd
network using automatic differentiation. PINNs solve inverse (1)
problems similarly to forward problems. We also present a with suitable boundary conditions (BCs) B(u, x) = 0 on
Python library for PINNs, DeepXDE. DeepXDE supports ∂Ω. For time-dependent problems, we consider time t as a
complex-geometry domains based on the technique of con- special component of x.
structive solid geometry, and enables the user code to be com-
pact, resembling closely the mathematical formulation. We PDE(λ)
introduce the usage of DeepXDE, and we also demonstrate ∂
the capability of PINNs and the user-friendliness of Deep- NN(x, t; θ) ∂t Tf
∂ û ∂ 2 û
∂t − λ ∂x2
XDE for two different examples. σ σ ∂2
∂x2
x σ σ
More recently, solving partial differential equations (PDEs) .. .. û Minimize
t . . I û(x, t) − gD (x, t) Loss θ∗
via deep learning has emerged as a potentially new sub-field
σ σ Tb
under the name of Scientific Machine Learning. To solve a ∂
∂n
∂ û
∂n (x, t) − gR (u, x, t)
PDE via deep learning, a key step is to constrain the neu- BC & IC
ral network to minimize the PDE residual, and several ap-
proaches have been proposed to accomplish this. Compared
to the traditional mesh-based methods, such as the finite dif- Figure 1: Schematic of a PINN for solving the diffusion
∂2u
ference method and the finite e lement m ethod, d eep learn- equation ∂u
∂t = λ ∂x2 with mixed BCs u(x, t) = gD (x, t)
∂u
ing could be a mesh-free approach by taking advantage of on ΓD ⊂ ∂Ω and ∂n (x, t) = gR (u, x, t) on ΓR ⊂ ∂Ω.
the automatic differentiation, and could break the curse of
dimensionality. Among these approaches, one could use the The algorithm of PINN is shown visually in the schematic
PDE in strong form directly; in this form, automatic differ- of Fig. 1 solving a diffusion equation. We explain each
entiation could be used directly to avoid truncation errors. step as follows. In a PINN, we first construct a neural net-
This approach is called physics-informed neural networks work û(x; θ) as a surrogate of the solution u(x). Here,
(PINNs). An attractive feature of PINNs is that it can be used θ = {W ` , b` }1≤`≤L is the set of all weight matrices and
to solve inverse problems similarly to forward problems. bias vectors in the network û. One advantage of choosing
In this paper, we present PINN algorithms implemented neural networks as the surrogate of u is that we can take the
in a Python library DeepXDE (https://github.com/lululxvi/ derivatives of û with respect to x by the automatic differen-
deepxde). DeepXDE can be used to solve multi-physics tiation. In the next step, we need to restrict û to satisfy the
problems, and supports complex-geometry domains based PDE and BCs. We only restrict û on some scattered points,
on the technique of constructive solid geometry (CSG), i.e., the training data T = {x1 , x2 , . . . , x|T | } of size |T |. T
hence avoiding tedious and time-consuming computational is comprised of two sets Tf ⊂ Ω and Tb ⊂ ∂Ω, which are the
geometry tasks. Last but not least, DeepXDE is designed to points in the domain and on the boundary, respectively. We
make the user code stay compact and manageable, resem- refer Tf and Tb as the sets of “residual points”. To measure
bling closely the mathematical formulation. the discrepancy between û and the constraints, we consider
the loss defined as:
Copyright c 2020, for this paper by its authors. Use permitted
under Creative Commons License Attribution 4.0 International L(θ; T ) = wf Lf (θ; Tf ) + wb Lb (θ; Tb ), (2)
(CCBY 4.0).
2
3 Demonstration example
where Lf (θ; Tf ) = |T1f | x∈Tf f x; ∂x ∂ û
P
1
,...;λ ,
2 We use PINNs and DeepXDE to solve inverse problems. A
1
Lb (θ; Tb ) = |Tb | x∈Tb kB(û, x)k22 , and wf and wb are the
P
diffusion-reaction system in porous media for the solute con-
weights. In the last step, the procedure of searching for a centrations CA , CB and CC (A + 2B → C) is described by
good θ by minimizing the loss L(θ; T ) using gradient-based ∂CA ∂ 2 CA 2 ∂CB ∂ 2 CB 2
optimizers is called “training”. ∂t = D ∂x2 − kf CA CB , ∂t = D ∂x2 − 2kf CA CB
for x ∈ [0, 1], t ∈ [0, 10] with IC CA (x, 0) = CB (x, 0) =
In inverse problems, there are some unknown parame- e−20x and BCs CA (0, t) = CB (0, t) = 1, CA (1, t) =
ters λ in Eq. (1), but we have extra information on points CB (1, t) = 0. We estimate the diffusion coefficient D =
Ti ⊂ Ω: I(u, x) = 0, for x ∈ Ti . PINNs solve inverse 2 × 10−3 and the reaction rate kf = 0.1 based on 40000
1
P by adding an2 extra term to Eq. (2): Li (θ, λ; Ti ) =
problems observations of the concentrations CA and CB in the spatio-
|Ti | x∈Ti kI(û, x)k2 . We then optimize θ and λ together: temporal domain. The identified D (1.98 × 10−3 ) and kf
∗ ∗
θ , λ = arg minθ,λ L(θ, λ; T ). (0.0971) are displayed in Fig. 3.
A 16 B 3
2 DeepXDE usage Figure 3: The
Parameter value
Parameter value
12 2 identified values of
In this section, we introduce the usage of DeepXDE. Deep- diffusion-reaction
8 True ρ Identified ρ 1
XDE makes the code stay compact and nice,
True σ resembling
Identified σ system converge
closely the mathematical formulation.
4 Solving
True β differential
Identified β 0 True kf Identified kf
True D Identified D to the true values
equations in DeepXDE is no more than specifying the prob- during the training
0 -1
0
lem using the build-in modules, including 1 computational
2 3 4 do- 5 6 0 1 2 3 4 5 6 7 8
# Iterations (104) # Iterations (104) process.
main (geometry and time), differential equations, ICs, BCs,
constraints, training data, network architecture, and training
hyperparameters. The workflow is shown in Procedure 1. Procedure 1 Usage of DeepXDE for solving differential
equations.
A B 1. Specify the computational domain using the geometry
- module.
A|B 2. Specify the differential equations using the grammar of
TensorFlow.
3. Specify the boundary and initial conditions.
A-B | &
4. Combine the geometry, PDE, and IC/BCs together into
data.PDE or data.TimePDE for time-independent or
A&B time-dependent problems, respectively. To specify train-
ing data, we can either set the specific point locations, or
Figure 2: CSG examples. (left) A and B represent the rectan- only set the number of points and then DeepXDE will
gle and circle, respectively. A|B, A − B, and A&B are con- sample the required number of points on a grid or ran-
structed from A and B. (right) A complex geometry (top) domly.
is constructed from a polygon, a rectangle and two circles 5. Construct a neural network using the maps module.
(bottom). 6. Define a Model by combining the PDE problem in Step
4 and the neural net in Step 5.
In DeepXDE, The built-in primitive geometries in- 7. Call Model.compile to set the optimization hyperpa-
clude interval, triangle, rectangle, polygon, rameters, such as optimizer and learning rate. The weights
disk, cuboid and sphere. Other geometries can be in Eq. (2) can be set here by loss weights.
constructed from these primitive geometries using three
boolean operations: union (|), difference (-) and 8. Call Model.train to train the network from random
intersection (&). This technique is called construc- initialization or a pre-trained model using the argument
tive solid geometry (CSG), see Fig. 2 for examples. model restore path. It is extremely flexible to mon-
DeepXDE supports four standard BCs, including itor and modify the training behavior using callbacks.
Dirichlet, Neumann, Robin, and Periodic, and 9. Call Model.predict to predict the PDE solution at
a more general BC can be defined using OperatorBC. different locations.
The initial condition can be defined using IC. There
are two networks available in DeepXDE: feed-forward
neural network (maps.FNN) and residual neural network
(maps.ResNet). It is also convenient to choose different
References
training hyperparameters, such as loss types, metrics, Lu, L.; Meng, X.; Mao, Z.; and Karniadakis, G. E. 2019.
optimizers, learning rate schedules, initializations and Deepxde: A deep learning library for solving differential
regularizations. equations. arXiv preprint arXiv:1907.04502.