Improved Training of Physics-Informed Neural Networks with Model
  Ensembles
  Learning the solution of partial differential equations (PDEs) with a neural
network (known in the literature as a physics-informed neural network, PINN) is
an attractive alternative to traditional solvers due to its elegancy, greater
flexibility and the ease of incorporating observed data. However, training
PINNs is notoriously difficult in practice. One problem is the existence of
multiple simple (but wrong) solutions which are attractive for PINNs when the
solution interval is too large. In this paper, we propose to expand the
solution interval gradually to make the PINN converge to the correct solution.
To find a good schedule for the solution interval expansion, we train an
ensemble of PINNs. The idea is that all ensemble members converge to the same
solution in the vicinity of observed data (e.g., initial conditions) while they
may be pulled towards different wrong solutions farther away from the
observations. Therefore, we use the ensemble agreement as the criterion for
including new points for computing the loss derived from PDEs. We show
experimentally that the proposed method can improve the accuracy of the found
solution.
