Neural Networks Trained to Solve Differential Equations Learn General
  Representations
  We introduce a technique based on the singular vector canonical correlation
analysis (SVCCA) for measuring the generality of neural network layers across a
continuously-parametrized set of tasks. We illustrate this method by studying
generality in neural networks trained to solve parametrized boundary value
problems based on the Poisson partial differential equation. We find that the
first hidden layer is general, and that deeper layers are successively more
specific. Next, we validate our method against an existing technique that
measures layer generality using transfer learning experiments. We find
excellent agreement between the two methods, and note that our method is much
faster, particularly for continuously-parametrized problems. Finally, we
visualize the general representations of the first layers, and interpret them
as generalized coordinates over the input domain.
