Boosting Ridge Regression for High Dimensional Data Classification
  Ridge regression is a well established regression estimator which can
conveniently be adapted for classification problems. One compelling reason is
probably the fact that ridge regression emits a closed-form solution thereby
facilitating the training phase. However in the case of high-dimensional
problems, the closed-form solution which involves inverting the regularised
covariance matrix is rather expensive to compute. The high computational demand
of such operation also renders difficulty in constructing ensemble of ridge
regressions. In this paper, we consider learning an ensemble of ridge
regressors where each regressor is trained in its own randomly projected
subspace. Subspace regressors are later combined via adaptive boosting
methodology. Experiments based on five high-dimensional classification problems
demonstrated the effectiveness of the proposed method in terms of learning time
and in some cases improved predictive performance can be observed.
