A Fast and Greedy Subset-of-Data (SoD) Scheme for Sparsification in
  Gaussian processes
  In their standard form Gaussian processes (GPs) provide a powerful
non-parametric framework for regression and classificaton tasks. Their one
limiting property is their $\mathcal{O}(N^{3})$ scaling where $N$ is the number
of training data points. In this paper we present a framework for GP training
with sequential selection of training data points using an intuitive selection
metric. The greedy forward selection strategy is devised to target two factors
- regions of high predictive uncertainty and underfit. Under this technique the
complexity of GP training is reduced to $\mathcal{O}(M^{3})$ where $(M \ll N)$
if $M$ data points (out of $N$) are eventually selected. The sequential nature
of the algorithm circumvents the need to invert the covariance matrix of
dimension $N \times N$ and enables the use of favourable matrix inverse update
identities. We outline the algorithm and sequential updates to the posterior
mean and variance. We demonstrate our method on selected one dimensional
functions and show that the loss in accuracy due to using a subset of data
points is marginal compared to the computational gains.
