On Robust Classification using Contractive Hamiltonian Neural ODEs
  Deep neural networks can be fragile and sensitive to small input
perturbations that might cause a significant change in the output. In this
paper, we employ contraction theory to improve the robustness of neural ODEs
(NODEs). A dynamical system is contractive if all solutions with different
initial conditions converge to each other asymptotically. As a consequence,
perturbations in initial conditions become less and less relevant over time.
Since in NODEs, the input data corresponds to the initial condition of
dynamical systems, we show contractivity can mitigate the effect of input
perturbations. More precisely, inspired by NODEs with Hamiltonian dynamics, we
propose a class of contractive Hamiltonian NODEs (CH-NODEs). By properly tuning
a scalar parameter, CH-NODEs ensure contractivity by design and can be trained
using standard backpropagation and gradient descent algorithms. Moreover,
CH-NODEs enjoy built-in guarantees of non-exploding gradients, which ensures a
well-posed training process. Finally, we demonstrate the robustness of CH-NODEs
on the MNIST image classification problem with noisy test datasets.
