Differential radial basis function network for sequence modelling
  We propose a differential radial basis function (RBF) network termed
RBF-DiffNet -- whose hidden layer blocks are partial differential equations
(PDEs) linear in terms of the RBF -- to make the baseline RBF network robust to
noise in sequential data. Assuming that the sequential data derives from the
discretisation of the solution to an underlying PDE, the differential RBF
network learns constant linear coefficients of the PDE, consequently
regularising the RBF network by following modified backward-Euler updates. We
experimentally validate the differential RBF network on the logistic map
chaotic timeseries as well as on 30 real-world timeseries provided by Walmart
in the M5 forecasting competition. The proposed model is compared with the
normalised and unnormalised RBF networks, ARIMA, and ensembles of multilayer
perceptrons (MLPs) and recurrent networks with long short-term memory (LSTM)
blocks. From the experimental results, RBF-DiffNet consistently shows a marked
reduction over the baseline RBF network in terms of the prediction error (e.g.,
26% reduction in the root mean squared scaled error on the M5 dataset);
RBF-DiffNet also shows a comparable performance to the LSTM ensemble at less
than one-sixteenth the LSTM computational time. Our proposed network
consequently enables more accurate predictions -- in the presence of
observational noise -- in sequence modelling tasks such as timeseries
forecasting that leverage the model interpretability, fast training, and
function approximation properties of the RBF network.
