Effective Discriminative Feature Selection with Non-trivial Solutions
  Feature selection and feature transformation, the two main ways to reduce
dimensionality, are often presented separately. In this paper, a feature
selection method is proposed by combining the popular transformation based
dimensionality reduction method Linear Discriminant Analysis (LDA) and sparsity
regularization. We impose row sparsity on the transformation matrix of LDA
through ${\ell}_{2,1}$-norm regularization to achieve feature selection, and
the resultant formulation optimizes for selecting the most discriminative
features and removing the redundant ones simultaneously. The formulation is
extended to the ${\ell}_{2,p}$-norm regularized case: which is more likely to
offer better sparsity when $0<p<1$. Thus the formulation is a better
approximation to the feature selection problem. An efficient algorithm is
developed to solve the ${\ell}_{2,p}$-norm based optimization problem and it is
proved that the algorithm converges when $0<p\le 2$. Systematical experiments
are conducted to understand the work of the proposed method. Promising
experimental results on various types of real-world data sets demonstrate the
effectiveness of our algorithm.
