Graph-Informed Neural Networks (GINNs) present a strategy for incorporating domain knowledge into scientific machine learning for complex physical systems. The construction utilizes probabilistic graphical models (PGMs) to incorporate expert knowledge, available data, constraints, etc. with physics-based models such as systems of ordinary and partial differential equations (ODEs and PDEs). Computationally intensive nodes in this hybrid model are replaced by the hidden nodes of a neural network (i.e., learned features). Once trained, the resulting GINN surrogate can cheaply generate physically-relevant predictions at scale thereby enabling robust sensitivity analysis and uncertainty quantification (UQ). As proof of concept, we build a GINN for a multiscale model of electrical double-layer capacitor dynamics embedded into a Bayesian network (BN) PDE hybrid model.
In recent years, several approaches have been proposed
to inform deep neural networks (DNNs) of physical laws
and constraints to ensure they produce physically sound
predictions. Two main classes of DNNs for building
surrogate representations of physics-based models described
by PDEs have emerged: physics-informed NNs (PINNs)
GINNs are particularly suited to enhance the computational workflow for complex systems featuring intrinsic computational bottlenecks and intricate physical relations among input CVs. Hence, to showcase the potential of this approach, we apply a GINN to simulation-based decisionmaking in electrical double-layer (EDL) supercapacitors, Copyright © 2021, for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CCBY 4.0). Domain- aware PGM r ω T cin lpor λD ϕΓ
ϕEDL χ± QuEEanDnvLitriFotoniermmseanottiaoflnInterest
Computational Bottleneck UOMppaseccrraoatliiDnniggf usion
Hybrid ...model (hig...h fidelitLSatyrtuecn)tture cin ConstrucCtonitronlVariablesand traϕEiDLniχn±g a GQuIantities of Interest g NN
Computational Bot leneck Simulation-based decision-making for design tasks involving complex multiscale/multiphysics systems requires predicting the impact of tunable CVs on the system’s QoIs. Typically, this is modeled by recasting the problem in a probabilistic framework where CVs and QoIs are represented as random quantities that can be sampled from their corresponding probability distributions. For most real-world applications, these are continuous, non-Gaussian variables that need to be characterized by their full PDF rather than through a finite set of moments.
Figure 1 visualizes the construction of a GINN surrogate Def − where it is deployed to build highly accurate kernel density estimators (KDEs) for the probability density functions (PDFs) of relevant output quantities of interest (QoIs).
GINN (surrogate model) . . .
. .
.
Macro Diffusion Structure Latent κef Def −
Def + t+ for a multiscale model of EDL supercapacitor dynamics. A BN, a type of directed acyclic PGM, systematically incorporates domain knowledge into the physics-based model through structured priors on CVs, resulting in a hybrid BN PDE model for macroscopic diffusion QoIs. The GINN retains the structured priors as inputs but replaces the hybrid model’s computationally intensive nodes, related to upscaling via homogenization, with learned features to speed up the generation of QoIs while maintaining physical relevance.
The GINN workflow, summarized in Fig. 2, consists of: 1. Data generation: Generate Nsam input-output (io) samples, divided into Ntrain training and Ntest test samples. 2. Training: Train the GINN with Ntrain training samples. 3. Testing: Test the trained GINN’s ability to handle unseen data using the Ntest test samples. 4. Repeat steps 1 through 3 (modifying Ntrain) until both the training and test error tolerance are satisfied. 5. Prediction: Draw Nsparmed inputs from the structured priors on the CVs and predict corresponding QoIs with the trained GINN surrogate. domain PGM structured knowledge priors (SP)
Physics-based PB (PB) or surrogate?
SP inputs
PB computations in, e.g., COMSOL, MATLAB, FENICS intervals for an equivalent computational cost (since learning the GINN’s parameters and predicting new data with the GINN carries a negligible computational expense). (GINNsurrogate) 2.5 2 1.5 1 0.5 0 0.1 0.12 0.1 0.08 0.06 0.04 0.02 00 (GINNsurrogate) 5 4 3 2 1 0.6 0 0.15 0.2 0.25 0.3 0.35 0.4 0.45 0.5 0.55 0.6 0.65 (Hybridmodel) 108 (Hybridmodel) (GINNsurrogate) 6 (GINNsurrogate) 4 2
Our full analysis, in
This work was performed while S. T. was employed by Stanford University. SURROGATE SP inputs
outputs PcoBmopuutpteudts? NO YES fuosriwngaridniptiraolpgaugeastsioens
for weights/biases
backpropagation learned weights/biases forward propagation
PcoBm opuutpteudts? NO YES forward propagation using learned weights/biases
NO GENERATING DATA
YES NO
PREDICTING DATA-DRIVEN UQ
YES modify NN hyperparameters NO TRAINING TESTING YES
GINN-based decision-making A GINN’s ability to cheaply generate io sample pairs can be leveraged to construct KDEs for the marginal and joint PDFs of QoIs with appropriate confidence intervals. Such nonparametric estimators form the building blocks for UQ tasks such as sensitivity analysis.
In Fig. 3, we plot KDEs for QoIs based on 8 103 samples simulated using the BN PDE (the minimum amount of io data needed to train the GINN) and on 107 samples predicted with the GINN. We find that the GINN-predicted KDEs do not include spurious features observed with the smaller, expensive-to-compute data set generated with the physics-based model, and achieve much tighter confidence