Local Shrunk Discriminant Analysis (LSDA)
  Dimensionality reduction is a crucial step for pattern recognition and data
mining tasks to overcome the curse of dimensionality. Principal component
analysis (PCA) is a traditional technique for unsupervised dimensionality
reduction, which is often employed to seek a projection to best represent the
data in a least-squares sense, but if the original data is nonlinear structure,
the performance of PCA will quickly drop. An supervised dimensionality
reduction algorithm called Linear discriminant analysis (LDA) seeks for an
embedding transformation, which can work well with Gaussian distribution data
or single-modal data, but for non-Gaussian distribution data or multimodal
data, it gives undesired results. What is worse, the dimension of LDA cannot be
more than the number of classes. In order to solve these issues, Local shrunk
discriminant analysis (LSDA) is proposed in this work to process the
non-Gaussian distribution data or multimodal data, which not only incorporate
both the linear and nonlinear structures of original data, but also learn the
pattern shrinking to make the data more flexible to fit the manifold structure.
Further, LSDA has more strong generalization performance, whose objective
function will become local LDA and traditional LDA when different extreme
parameters are utilized respectively. What is more, a new efficient
optimization algorithm is introduced to solve the non-convex objective function
with low computational cost. Compared with other related approaches, such as
PCA, LDA and local LDA, the proposed method can derive a subspace which is more
suitable for non-Gaussian distribution and real data. Promising experimental
results on different kinds of data sets demonstrate the effectiveness of the
proposed approach.
