Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks
Physics-informed neural networks (PINNs) leverage neural-networks to find the
solutions of partial differential equation (PDE)-constrained optimization
problems with initial conditions and boundary conditions as soft constraints.
These soft constraints are often considered to be the sources of the complexity
in the training phase of PINNs. Here, we demonstrate that the challenge of
training (i) persists even when the boundary conditions are strictly enforced,
and (ii) is closely related to the Kolmogorov n-width associated with problems
demonstrating transport, convection, traveling waves, or moving fronts. Given
this realization, we describe the mechanism underlying the training schemes
such as those used in eXtended PINNs (XPINN), curriculum regularization, and
sequence-to-sequence learning. For an important category of PDEs, i.e.,
governed by non-linear convection-diffusion equation, we propose reformulating
PINNs on a Lagrangian frame of reference, i.e., LPINNs, as a PDE-informed
solution. A parallel architecture with two branches is proposed. One branch
solves for the state variables on the characteristics, and the second branch
solves for the low-dimensional characteristics curves. The proposed
architecture conforms to the causality innate to the convection, and leverages
the direction of travel of the information in the domain. Finally, we
demonstrate that the loss landscapes of LPINNs are less sensitive to the
so-called "complexity" of the problems, compared to those in the traditional
PINNs in the Eulerian framework.