A High-Performance Implementation of Bayesian Matrix Factorization with
  Limited Communication
  Matrix factorization is a very common machine learning technique in
recommender systems. Bayesian Matrix Factorization (BMF) algorithms would be
attractive because of their ability to quantify uncertainty in their
predictions and avoid over-fitting, combined with high prediction accuracy.
However, they have not been widely used on large-scale data because of their
prohibitive computational cost. In recent work, efforts have been made to
reduce the cost, both by improving the scalability of the BMF algorithm as well
as its implementation, but so far mainly separately. In this paper we show that
the state-of-the-art of both approaches to scalability can be combined. We
combine the recent highly-scalable Posterior Propagation algorithm for BMF,
which parallelizes computation of blocks of the matrix, with a distributed BMF
implementation that users asynchronous communication within each block. We show
that the combination of the two methods gives substantial improvements in the
scalability of BMF on web-scale datasets, when the goal is to reduce the
wall-clock time.
