Time-Adaptive Recurrent Neural Networks
  Data are often sampled irregularly in time. Dealing with this using Recurrent
Neural Networks (RNNs) traditionally involved ignoring the fact, feeding the
time differences as additional inputs, or resampling the data. All these
methods have their shortcomings. We propose an elegant alternative approach
where instead the RNN is in effect resampled in time to match the time of the
data. We use Echo State Network (ESN) and Gated Recurrent Unit (GRU) as the
basis for our solution. Such RNNs can be seen as discretizations of
continuous-time dynamical systems, which gives a solid theoretical ground for
our approach. Similar recent observations have been made in feed-forward neural
networks as neural ordinary differential equations. Our Time-Adaptive ESN
(TAESN) and GRU (TAGRU) models allow for a direct model time setting and
require no additional training, parameter tuning, or computation compared to
the regular counterparts, thus retaining their original efficiency. We confirm
empirically that our models can effectively compensate for the
time-non-uniformity of the data and demonstrate that they compare favorably to
data resampling, classical RNN methods, and alternative RNN models proposed to
deal with time irregularities on several real-world nonuniform-time datasets.
