Training multi-objective/multi-task collocation physics-informed neural
  network with student/teachers transfer learnings
  This paper presents a PINN training framework that employs (1) pre-training
steps that accelerates and improve the robustness of the training of
physics-informed neural network with auxiliary data stored in point clouds, (2)
a net-to-net knowledge transfer algorithm that improves the weight
initialization of the neural network and (3) a multi-objective optimization
algorithm that may improve the performance of a physical-informed neural
network with competing constraints. We consider the training and transfer and
multi-task learning of physics-informed neural network (PINN) as
multi-objective problems where the physics constraints such as the governing
equation, boundary conditions, thermodynamic inequality, symmetry, and
invariant properties, as well as point cloud used for pre-training can
sometimes lead to conflicts and necessitating the seek of the Pareto optimal
solution. In these situations, weighted norms commonly used to handle multiple
constraints may lead to poor performance, while other multi-objective
algorithms may scale poorly with increasing dimensionality. To overcome this
technical barrier, we adopt the concept of vectorized objective function and
modify a gradient descent approach to handle the issue of conflicting
gradients. Numerical experiments are compared the benchmark boundary value
problems solved via PINN. The performance of the proposed paradigm is compared
against the classical equal-weighted norm approach. Our numerical experiments
indicate that the brittleness and lack of robustness demonstrated in some PINN
implementations can be overcome with the proposed strategy.
