Learning Physics-Informed Neural Networks without Stacked
  Back-propagation
  Physics-Informed Neural Network (PINN) has become a commonly used machine
learning approach to solve partial differential equations (PDE). But, facing
high-dimensional second-order PDE problems, PINN will suffer from severe
scalability issues since its loss includes second-order derivatives, the
computational cost of which will grow along with the dimension during stacked
back-propagation. In this paper, we develop a novel approach that can
significantly accelerate the training of Physics-Informed Neural Networks. In
particular, we parameterize the PDE solution by the Gaussian smoothed model and
show that, derived from Stein's Identity, the second-order derivatives can be
efficiently calculated without back-propagation. We further discuss the model
capacity and provide variance reduction methods to address key limitations in
the derivative estimation. Experimental results show that our proposed method
can achieve competitive error compared to standard PINN training but is two
orders of magnitude faster.
