Boosting through Optimization of Margin Distributions
  Boosting has attracted much research attention in the past decade. The
success of boosting algorithms may be interpreted in terms of the margin
theory. Recently it has been shown that generalization error of classifiers can
be obtained by explicitly taking the margin distribution of the training data
into account. Most of the current boosting algorithms in practice usually
optimizes a convex loss function and do not make use of the margin
distribution. In this work we design a new boosting algorithm, termed
margin-distribution boosting (MDBoost), which directly maximizes the average
margin and minimizes the margin variance simultaneously. This way the margin
distribution is optimized. A totally-corrective optimization algorithm based on
column generation is proposed to implement MDBoost. Experiments on UCI datasets
show that MDBoost outperforms AdaBoost and LPBoost in most cases.
