Diffusion Nets
  Non-linear manifold learning enables high-dimensional data analysis, but
requires out-of-sample-extension methods to process new data points. In this
paper, we propose a manifold learning algorithm based on deep learning to
create an encoder, which maps a high-dimensional dataset and its
low-dimensional embedding, and a decoder, which takes the embedded data back to
the high-dimensional space. Stacking the encoder and decoder together
constructs an autoencoder, which we term a diffusion net, that performs
out-of-sample-extension as well as outlier detection. We introduce new neural
net constraints for the encoder, which preserves the local geometry of the
points, and we prove rates of convergence for the encoder. Also, our approach
is efficient in both computational complexity and memory requirements, as
opposed to previous methods that require storage of all training points in both
the high-dimensional and the low-dimensional spaces to calculate the
out-of-sample-extension and the pre-image.
