Progressive Growing of Neural ODEs
  Neural Ordinary Differential Equations (NODEs) have proven to be a powerful
modeling tool for approximating (interpolation) and forecasting (extrapolation)
irregularly sampled time series data. However, their performance degrades
substantially when applied to real-world data, especially long-term data with
complex behaviors (e.g., long-term trend across years, mid-term seasonality
across months, and short-term local variation across days). To address the
modeling of such complex data with different behaviors at different frequencies
(time spans), we propose a novel progressive learning paradigm of NODEs for
long-term time series forecasting. Specifically, following the principle of
curriculum learning, we gradually increase the complexity of data and network
capacity as training progresses. Our experiments with both synthetic data and
real traffic data (PeMS Bay Area traffic data) show that our training
methodology consistently improves the performance of vanilla NODEs by over 64%.
