Long-time predictive modeling of nonlinear dynamical systems using
  neural networks
  We study the use of feedforward neural networks (FNN) to develop models of
nonlinear dynamical systems from data. Emphasis is placed on predictions at
long times, with limited data availability. Inspired by global stability
analysis, and the observation of the strong correlation between the local error
and the maximum singular value of the Jacobian of the ANN, we introduce
Jacobian regularization in the loss function. This regularization suppresses
the sensitivity of the prediction to the local error and is shown to improve
accuracy and robustness. Comparison between the proposed approach and sparse
polynomial regression is presented in numerical examples ranging from simple
ODE systems to nonlinear PDE systems including vortex shedding behind a
cylinder, and instability-driven buoyant mixing flow. Furthermore, limitations
of feedforward neural networks are highlighted, especially when the training
data does not include a low dimensional attractor. Strategies of data
augmentation are presented as remedies to address these issues to a certain
extent.
