Locally Linear Embedding and its Variants: Tutorial and Survey
  This is a tutorial and survey paper for Locally Linear Embedding (LLE) and
its variants. The idea of LLE is fitting the local structure of manifold in the
embedding space. In this paper, we first cover LLE, kernel LLE, inverse LLE,
and feature fusion with LLE. Then, we cover out-of-sample embedding using
linear reconstruction, eigenfunctions, and kernel mapping. Incremental LLE is
explained for embedding streaming data. Landmark LLE methods using the Nystrom
approximation and locally linear landmarks are explained for big data
embedding. We introduce the methods for parameter selection of number of
neighbors using residual variance, Procrustes statistics, preservation
neighborhood error, and local neighborhood selection. Afterwards, Supervised
LLE (SLLE), enhanced SLLE, SLLE projection, probabilistic SLLE, supervised
guided LLE (using Hilbert-Schmidt independence criterion), and semi-supervised
LLE are explained for supervised and semi-supervised embedding. Robust LLE
methods using least squares problem and penalty functions are also introduced
for embedding in the presence of outliers and noise. Then, we introduce fusion
of LLE with other manifold learning methods including Isomap (i.e., ISOLLE),
principal component analysis, Fisher discriminant analysis, discriminant LLE,
and Isotop. Finally, we explain weighted LLE in which the distances,
reconstruction weights, or the embeddings are adjusted for better embedding; we
cover weighted LLE for deformed distributed data, weighted LLE using
probability of occurrence, SLLE by adjusting weights, modified LLE, and
iterative LLE.
