Roweis Discriminant Analysis: A Generalized Subspace Learning Method
  We present a new method which generalizes subspace learning based on
eigenvalue and generalized eigenvalue problems. This method, Roweis
Discriminant Analysis (RDA), is named after Sam Roweis to whom the field of
subspace learning owes significantly. RDA is a family of infinite number of
algorithms where Principal Component Analysis (PCA), Supervised PCA (SPCA), and
Fisher Discriminant Analysis (FDA) are special cases. One of the extreme
special cases, which we name Double Supervised Discriminant Analysis (DSDA),
uses the labels twice; it is novel and has not appeared elsewhere. We propose a
dual for RDA for some special cases. We also propose kernel RDA, generalizing
kernel PCA, kernel SPCA, and kernel FDA, using both dual RDA and representation
theory. Our theoretical analysis explains previously known facts such as why
SPCA can use regression but FDA cannot, why PCA and SPCA have duals but FDA
does not, why kernel PCA and kernel SPCA use kernel trick but kernel FDA does
not, and why PCA is the best linear method for reconstruction. Roweisfaces and
kernel Roweisfaces are also proposed generalizing eigenfaces, Fisherfaces,
supervised eigenfaces, and their kernel variants. We also report experiments
showing the effectiveness of RDA and kernel RDA on some benchmark datasets.
