A Comparison Study on Nonlinear Dimension Reduction Methods with Kernel
  Variations: Visualization, Optimization and Classification
  Because of high dimensionality, correlation among covariates, and noise
contained in data, dimension reduction (DR) techniques are often employed to
the application of machine learning algorithms. Principal Component Analysis
(PCA), Linear Discriminant Analysis (LDA), and their kernel variants (KPCA,
KLDA) are among the most popular DR methods. Recently, Supervised Kernel
Principal Component Analysis (SKPCA) has been shown as another successful
alternative. In this paper, brief reviews of these popular techniques are
presented first. We then conduct a comparative performance study based on three
simulated datasets, after which the performance of the techniques are evaluated
through application to a pattern recognition problem in face image analysis.
The gender classification problem is considered on MORPH-II and FG-NET, two
popular longitudinal face aging databases. Several feature extraction methods
are used, including biologically-inspired features (BIF), local binary patterns
(LBP), histogram of oriented gradients (HOG), and the Active Appearance Model
(AAM). After applications of DR methods, a linear support vector machine (SVM)
is deployed with gender classification accuracy rates exceeding 95% on
MORPH-II, competitive with benchmark results. A parallel computational approach
is also proposed, attaining faster processing speeds and similar recognition
rates on MORPH-II. Our computational approach can be applied to practical
gender classification systems and generalized to other face analysis tasks,
such as race classification and age prediction.
