Noether Networks: Meta-Learning Useful Conserved Quantities
  Progress in machine learning (ML) stems from a combination of data
availability, computational resources, and an appropriate encoding of inductive
biases. Useful biases often exploit symmetries in the prediction problem, such
as convolutional networks relying on translation equivariance. Automatically
discovering these useful symmetries holds the potential to greatly improve the
performance of ML systems, but still remains a challenge. In this work, we
focus on sequential prediction problems and take inspiration from Noether's
theorem to reduce the problem of finding inductive biases to meta-learning
useful conserved quantities. We propose Noether Networks: a new type of
architecture where a meta-learned conservation loss is optimized inside the
prediction function. We show, theoretically and experimentally, that Noether
Networks improve prediction quality, providing a general framework for
discovering inductive biases in sequential problems.
