Learning Manifolds from Non-stationary Streaming Data
  Streaming adaptations of manifold learning based dimensionality reduction
methods, such as Isomap, are based on the assumption that a small initial batch
of observations is enough for exact learning of the manifold, while remaining
streaming data instances can be cheaply mapped to this manifold. However, there
are no theoretical results to show that this core assumption is valid.
Moreover, such methods typically assume that the underlying data distribution
is stationary. Such methods are not equipped to detect, or handle, sudden
changes or gradual drifts in the distribution that may occur when the data is
streaming. We present theoretical results to show that the quality of a
manifold asymptotically converges as the size of data increases. We then show
that a Gaussian Process Regression (GPR) model, that uses a manifold-specific
kernel function and is trained on an initial batch of sufficient size, can
closely approximate the state-of-art streaming Isomap algorithms. The
predictive variance obtained from the GPR prediction is then shown to be an
effective detector of changes in the underlying data distribution. Results on
several synthetic and real data sets show that the resulting algorithm can
effectively learn lower dimensional representation of high dimensional data in
a streaming setting, while identifying shifts in the generative distribution.
