Neural Ordinary Differential Equation based Recurrent Neural Network
  Model
  Neural differential equations are a promising new member in the neural
network family. They show the potential of differential equations for time
series data analysis. In this paper, the strength of the ordinary differential
equation (ODE) is explored with a new extension. The main goal of this work is
to answer the following questions: (i)~can ODE be used to redefine the existing
neural network model? (ii)~can Neural ODEs solve the irregular sampling rate
challenge of existing neural network models for a continuous time series, i.e.,
length and dynamic nature, (iii)~how to reduce the training and evaluation time
of existing Neural ODE systems? This work leverages the mathematical foundation
of ODEs to redesign traditional RNNs such as Long Short-Term Memory (LSTM) and
Gated Recurrent Unit (GRU). The main contribution of this paper is to
illustrate the design of two new ODE-based RNN models (GRU-ODE model and
LSTM-ODE) which can compute the hidden state and cell state at any point of
time using an ODE solver. These models reduce the computation overhead of
hidden state and cell state by a vast amount. The performance evaluation of
these two new models for learning continuous time series with irregular
sampling rate is then demonstrated. Experiments show that these new ODE based
RNN models require less training time than Latent ODEs and conventional Neural
ODEs. They can achieve higher accuracy quickly, and the design of the neural
network is simpler than, previous neural ODE systems.
