STEER: Simple Temporal Regularization For Neural ODEs
  Training Neural Ordinary Differential Equations (ODEs) is often
computationally expensive. Indeed, computing the forward pass of such models
involves solving an ODE which can become arbitrarily complex during training.
Recent works have shown that regularizing the dynamics of the ODE can partially
alleviate this. In this paper we propose a new regularization technique:
randomly sampling the end time of the ODE during training. The proposed
regularization is simple to implement, has negligible overhead and is effective
across a wide variety of tasks. Further, the technique is orthogonal to several
other methods proposed to regularize the dynamics of ODEs and as such can be
used in conjunction with them. We show through experiments on normalizing
flows, time series models and image recognition that the proposed
regularization can significantly decrease training time and even improve
performance over baseline models.
