Early stopping for kernel boosting algorithms: A general analysis with
  localized complexities
  Early stopping of iterative algorithms is a widely-used form of
regularization in statistics, commonly used in conjunction with boosting and
related gradient-type algorithms. Although consistency results have been
established in some settings, such estimators are less well-understood than
their analogues based on penalized regularization. In this paper, for a
relatively broad class of loss functions and boosting algorithms (including
L2-boost, LogitBoost and AdaBoost, among others), we exhibit a direct
connection between the performance of a stopped iterate and the localized
Gaussian complexity of the associated function class. This connection allows us
to show that local fixed point analysis of Gaussian or Rademacher complexities,
now standard in the analysis of penalized estimators, can be used to derive
optimal stopping rules. We derive such stopping rules in detail for various
kernel classes, and illustrate the correspondence of our theory with practice
for Sobolev kernel classes.
