A Gaussian Belief Propagation Solver for Large Scale Support Vector
  Machines
  Support vector machines (SVMs) are an extremely successful type of
classification and regression algorithms. Building an SVM entails solving a
constrained convex quadratic programming problem, which is quadratic in the
number of training samples. We introduce an efficient parallel implementation
of an support vector regression solver, based on the Gaussian Belief
Propagation algorithm (GaBP).
  In this paper, we demonstrate that methods from the complex system domain
could be utilized for performing efficient distributed computation. We compare
the proposed algorithm to previously proposed distributed and single-node SVM
solvers. Our comparison shows that the proposed algorithm is just as accurate
as these solvers, while being significantly faster, especially for large
datasets. We demonstrate scalability of the proposed algorithm to up to 1,024
computing nodes and hundreds of thousands of data points using an IBM Blue Gene
supercomputer. As far as we know, our work is the largest parallel
implementation of belief propagation ever done, demonstrating the applicability
of this algorithm for large scale distributed computing systems.
