From Kernel Machines to Ensemble Learning
  Ensemble methods such as boosting combine multiple learners to obtain better
prediction than could be obtained from any individual learner. Here we propose
a principled framework for directly constructing ensemble learning methods from
kernel methods. Unlike previous studies showing the equivalence between
boosting and support vector machines (SVMs), which needs a translation
procedure, we show that it is possible to design boosting-like procedure to
solve the SVM optimization problems.
  In other words, it is possible to design ensemble methods directly from SVM
without any middle procedure.
  This finding not only enables us to design new ensemble learning methods
directly from kernel methods, but also makes it possible to take advantage of
those highly-optimized fast linear SVM solvers for ensemble learning.
  We exemplify this framework for designing binary ensemble learning as well as
a new multi-class ensemble learning methods.
  Experimental results demonstrate the flexibility and usefulness of the
proposed framework.
