Distributed Optimization with Arbitrary Local Solvers
  With the growth of data and necessity for distributed optimization methods,
solvers that work well on a single machine must be re-designed to leverage
distributed computation. Recent work in this area has been limited by focusing
heavily on developing highly specific methods for the distributed environment.
These special-purpose methods are often unable to fully leverage the
competitive performance of their well-tuned and customized single machine
counterparts. Further, they are unable to easily integrate improvements that
continue to be made to single machine methods. To this end, we present a
framework for distributed optimization that both allows the flexibility of
arbitrary solvers to be used on each (single) machine locally, and yet
maintains competitive performance against other state-of-the-art
special-purpose distributed methods. We give strong primal-dual convergence
rate guarantees for our framework that hold for arbitrary local solvers. We
demonstrate the impact of local solver selection both theoretically and in an
extensive experimental comparison. Finally, we provide thorough implementation
details for our framework, highlighting areas for practical performance gains.
