Asymptotic Analysis of an Ensemble of Randomly Projected Linear
  Discriminants
  Datasets from the fields of bioinformatics, chemometrics, and face
recognition are typically characterized by small samples of high-dimensional
data. Among the many variants of linear discriminant analysis that have been
proposed in order to rectify the issues associated with classification in such
a setting, the classifier in [1], composed of an ensemble of randomly projected
linear discriminants, seems especially promising; it is computationally
efficient and, with the optimal projection dimension parameter setting, is
competitive with the state-of-the-art. In this work, we seek to further
understand the behavior of this classifier through asymptotic analysis. Under
the assumption of a growth regime in which the dataset and projection
dimensions grow at constant rates to each other, we use random matrix theory to
derive asymptotic misclassification probabilities showing the effect of the
ensemble as a regularization of the data sample covariance matrix. The
asymptotic errors further help to identify situations in which the ensemble
offers a performance advantage. We also develop a consistent estimator of the
misclassification probability as an alternative to the computationally-costly
cross-validation estimator, which is conventionally used for parameter tuning.
Finally, we demonstrate the use of our estimator for tuning the projection
dimension on both real and synthetic data.
