We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

cs.LG

Change to browse by:

References & Citations

DBLP - CS Bibliography

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Computer Science > Machine Learning

Title: Critical Investigation of Failure Modes in Physics-informed Neural Networks

Abstract: Several recent works in scientific machine learning have revived interest in the application of neural networks to partial differential equations (PDEs). A popular approach is to aggregate the residual form of the governing PDE and its boundary conditions as soft penalties into a composite objective/loss function for training neural networks, which is commonly referred to as physics-informed neural networks (PINNs). In the present study, we visualize the loss landscapes and distributions of learned parameters and explain the ways this particular formulation of the objective function may hinder or even prevent convergence when dealing with challenging target solutions. We construct a purely data-driven loss function composed of both the boundary loss and the domain loss. Using this data-driven loss function and, separately, a physics-informed loss function, we then train two neural network models with the same architecture. We show that incomparable scales between boundary and domain loss terms are the culprit behind the poor performance. Additionally, we assess the performance of both approaches on two elliptic problems with increasingly complex target solutions. Based on our analysis of their loss landscapes and learned parameter distributions, we observe that a physics-informed neural network with a composite objective function formulation produces highly non-convex loss surfaces that are difficult to optimize and are more prone to the problem of vanishing gradients.
Subjects: Machine Learning (cs.LG); Numerical Analysis (math.NA)
Journal reference: AIAA SCITECH 2022 Forum (p. 2353)
DOI: 10.2514/6.2022-2353
Cite as: arXiv:2206.09961 [cs.LG]
  (or arXiv:2206.09961v2 [cs.LG] for this version)

Submission history

From: Shamsulhaq Basir [view email]
[v1] Mon, 20 Jun 2022 18:43:35 GMT (2813kb,D)
[v2] Tue, 28 Jun 2022 04:06:09 GMT (2814kb,D)

Link back to: arXiv, form interface, contact.