Mathematical Analysis on Out-of-Sample Extensions
  Let $X=\mathbf{X}\cup\mathbf{Z}$ be a data set in $\mathbb{R}^D$, where
$\mathbf{X}$ is the training set and $\mathbf{Z}$ is the test one. Many
unsupervised learning algorithms based on kernel methods have been developed to
provide dimensionality reduction (DR) embedding for a given training set $\Phi:
\mathbf{X} \to \mathbb{R}^d$ ( $d\ll D$) that maps the high-dimensional data
$\mathbf{X}$ to its low-dimensional feature representation
$\mathbf{Y}=\Phi(\mathbf{X})$. However, these algorithms do not
straightforwardly produce DR of the test set $\mathbf{Z}$. An out-of-sample
extension method provides DR of $\mathbf{Z}$ using an extension of the existent
embedding $\Phi$, instead of re-computing the DR embedding for the whole set
$X$. Among various out-of-sample DR extension methods, those based on
Nystr\"{o}m approximation are very attractive. Many papers have developed such
out-of-extension algorithms and shown their validity by numerical experiments.
However, the mathematical theory for the DR extension still need further
consideration. Utilizing the reproducing kernel Hilbert space (RKHS) theory,
this paper develops a preliminary mathematical analysis on the out-of-sample DR
extension operators. It treats an out-of-sample DR extension operator as an
extension of the identity on the RKHS defined on $\mathbf{X}$. Then the
Nystr\"{o}m-type DR extension turns out to be an orthogonal projection. In the
paper, we also present the conditions for the exact DR extension and give the
estimate for the error of the extension.
