A case for new neural network smoothness constraints
  How sensitive should machine learning models be to input changes? We tackle
the question of model smoothness and show that it is a useful inductive bias
which aids generalization, adversarial robustness, generative modeling and
reinforcement learning. We explore current methods of imposing smoothness
constraints and observe they lack the flexibility to adapt to new tasks, they
don't account for data modalities, they interact with losses, architectures and
optimization in ways not yet fully understood. We conclude that new advances in
the field are hinging on finding ways to incorporate data, tasks and learning
into our definitions of smoothness.
