Neural Galerkin Scheme with Active Learning for High-Dimensional
  Evolution Equations
  Deep neural networks have been shown to provide accurate function
approximations in high dimensions. However, fitting network parameters requires
training data that may not be available beforehand, which is particularly
challenging in science and engineering applications where often it is even
unclear how to collect new informative training data in the first place. This
work proposes Neural Galerkin schemes based on deep learning that generate
training data samples with active learning for numerically solving
high-dimensional partial differential equations. Neural Galerkin schemes train
networks by minimizing the residual sequentially over time, which enables
adaptively collecting new training data in a self-informed manner that is
guided by the dynamics described by the partial differential equations, which
is in stark contrast to many other machine learning methods that aim to fit
network parameters globally in time without taking into account training data
acquisition. Our finding is that the active form of gathering training data of
the proposed Neural Galerkin schemes is key for numerically realizing the
expressive power of networks in high dimensions. Numerical experiments
demonstrate that Neural Galerkin schemes have the potential to enable
simulating phenomena and processes with many variables for which traditional
and other deep-learning-based solvers fail, especially when features of the
solutions evolve locally such as in high-dimensional wave propagation problems
and interacting particle systems described by Fokker-Planck and kinetic
equations.
