Simultaneously Learning Neighborship and Projection Matrix for
  Supervised Dimensionality Reduction
  Explicitly or implicitly, most of dimensionality reduction methods need to
determine which samples are neighbors and the similarity between the neighbors
in the original highdimensional space. The projection matrix is then learned on
the assumption that the neighborhood information (e.g., the similarity) is
known and fixed prior to learning. However, it is difficult to precisely
measure the intrinsic similarity of samples in high-dimensional space because
of the curse of dimensionality. Consequently, the neighbors selected according
to such similarity might and the projection matrix obtained according to such
similarity and neighbors are not optimal in the sense of classification and
generalization. To overcome the drawbacks, in this paper we propose to let the
similarity and neighbors be variables and model them in low-dimensional space.
Both the optimal similarity and projection matrix are obtained by minimizing a
unified objective function. Nonnegative and sum-to-one constraints on the
similarity are adopted. Instead of empirically setting the regularization
parameter, we treat it as a variable to be optimized. It is interesting that
the optimal regularization parameter is adaptive to the neighbors in
low-dimensional space and has intuitive meaning. Experimental results on the
YALE B, COIL-100, and MNIST datasets demonstrate the effectiveness of the
proposed method.
