Neural Jump Ordinary Differential Equations: Consistent Continuous-Time
  Prediction and Filtering
  Combinations of neural ODEs with recurrent neural networks (RNN), like
GRU-ODE-Bayes or ODE-RNN are well suited to model irregularly observed time
series. While those models outperform existing discrete-time approaches, no
theoretical guarantees for their predictive capabilities are available.
Assuming that the irregularly-sampled time series data originates from a
continuous stochastic process, the $L^2$-optimal online prediction is the
conditional expectation given the currently available information. We introduce
the Neural Jump ODE (NJ-ODE) that provides a data-driven approach to learn,
continuously in time, the conditional expectation of a stochastic process. Our
approach models the conditional expectation between two observations with a
neural ODE and jumps whenever a new observation is made. We define a novel
training framework, which allows us to prove theoretical guarantees for the
first time. In particular, we show that the output of our model converges to
the $L^2$-optimal prediction. This can be interpreted as solution to a special
filtering problem. We provide experiments showing that the theoretical results
also hold empirically. Moreover, we experimentally show that our model
outperforms the baselines in more complex learning tasks and give comparisons
on real-world datasets.
