Neural Controlled Differential Equations for Online Prediction Tasks
  Neural controlled differential equations (Neural CDEs) are a continuous-time
extension of recurrent neural networks (RNNs), achieving state-of-the-art
(SOTA) performance at modelling functions of irregular time series. In order to
interpret discrete data in continuous time, current implementations rely on
non-causal interpolations of the data. This is fine when the whole time series
is observed in advance, but means that Neural CDEs are not suitable for use in
\textit{online prediction tasks}, where predictions need to be made in
real-time: a major use case for recurrent networks. Here, we show how this
limitation may be rectified. First, we identify several theoretical conditions
that interpolation schemes for Neural CDEs should satisfy, such as boundedness
and uniqueness. Second, we use these to motivate the introduction of new
schemes that address these conditions, offering in particular measurability
(for online prediction), and smoothness (for speed). Third, we empirically
benchmark our online Neural CDE model on three continuous monitoring tasks from
the MIMIC-IV medical database: we demonstrate improved performance on all tasks
against ODE benchmarks, and on two of the three tasks against SOTA non-ODE
benchmarks.
