Online Learning with Radial Basis Function Networks
  We experiment with the log-returns of financial time series, providing
multi-horizon forecasts with a selection of robust supervised learners. We
devise an external input selection algorithm that aims to maximise regression
R^2 whilst minimising feature correlations and can operate efficiently in a
high-dimensional setting. We improve upon the earlier work on radial basis
function networks (rbfnets), which applies feature representation transfer from
clustering algorithms to supervised learners. Rather than using a randomised,
scalar standard deviation for each hidden processing unit (hpu)'s radial basis
function, we use a covariance matrix estimated via a Bayesian map approach. If
many (few) training data points are assigned to the j'th cluster, the j'th
covariance matrix will resemble the maximum likelihood (diagonalised variance
prior) estimate. More precisely, we operate with and adapt the precision
(inverse covariance) matrices; this leads to a test time fitting
time-complexity of O(kd^2), where k is the number of hpus and d is the external
input dimensionality. Our approach leads to a reduction from O(kd^3) when
inverting covariance matrices. Furthermore, we sequentially optimise the hpu
parameters with an exponential decay to facilitate regime changes or concept
drifts. Our experiments demonstrate that our online rbfnet outperforms a
random-walk baseline and several powerful batch learners. The outperformance is
not purely down to sequential updating in the test set. Instead, a competitor
ewrls model is updated similarly and performs less well than several
batch-learners. Finally, our online rbfnet has hpus that retain greater
similarity between training and test vectors, ensuring that it obtains the
smallest prediction mean squared errors.
