Two-Manifold Problems
  Recently, there has been much interest in spectral approaches to learning
manifolds---so-called kernel eigenmap methods. These methods have had some
successes, but their applicability is limited because they are not robust to
noise. To address this limitation, we look at two-manifold problems, in which
we simultaneously reconstruct two related manifolds, each representing a
different view of the same data. By solving these interconnected learning
problems together and allowing information to flow between them, two-manifold
algorithms are able to succeed where a non-integrated approach would fail: each
view allows us to suppress noise in the other, reducing bias in the same way
that an instrumental variable allows us to remove bias in a {linear}
dimensionality reduction problem. We propose a class of algorithms for
two-manifold problems, based on spectral decomposition of cross-covariance
operators in Hilbert space. Finally, we discuss situations where two-manifold
problems are useful, and demonstrate that solving a two-manifold problem can
aid in learning a nonlinear dynamical system from limited data.
