Stabilized Neural Ordinary Differential Equations for Long-Time
  Forecasting of Dynamical Systems
  In data-driven modeling of spatiotemporal phenomena careful consideration
often needs to be made in capturing the dynamics of the high wavenumbers. This
problem becomes especially challenging when the system of interest exhibits
shocks or chaotic dynamics. We present a data-driven modeling method that
accurately captures shocks and chaotic dynamics by proposing a novel
architecture, stabilized neural ordinary differential equation (ODE). In our
proposed architecture, we learn the right-hand-side (RHS) of an ODE by adding
the outputs of two NN together where one learns a linear term and the other a
nonlinear term. Specifically, we implement this by training a sparse linear
convolutional NN to learn the linear term and a dense fully-connected nonlinear
NN to learn the nonlinear term. This is in contrast with the standard neural
ODE which involves training only a single NN for learning the RHS. We apply
this setup to the viscous Burgers equation, which exhibits shocked behavior,
and show better short-time tracking and prediction of the energy spectrum at
high wavenumbers than a standard neural ODE. We also find that the stabilized
neural ODE models are much more robust to noisy initial conditions than the
standard neural ODE approach. We also apply this method to chaotic trajectories
of the Kuramoto-Sivashinsky equation. In this case, stabilized neural ODEs keep
long-time trajectories on the attractor, and are highly robust to noisy initial
conditions, while standard neural ODEs fail at achieving either of these
results. We conclude by demonstrating how stabilizing neural ODEs provide a
natural extension for use in reduced-order modeling by projecting the dynamics
onto the eigenvectors of the learned linear term.
