Decentralized Composite Optimization with Compression
  Decentralized optimization and communication compression have exhibited their
great potential in accelerating distributed machine learning by mitigating the
communication bottleneck in practice. While existing decentralized algorithms
with communication compression mostly focus on the problems with only smooth
components, we study the decentralized stochastic composite optimization
problem with a potentially non-smooth component. A \underline{Prox}imal
gradient \underline{L}in\underline{EA}r convergent \underline{D}ecentralized
algorithm with compression, Prox-LEAD, is proposed with rigorous theoretical
analyses in the general stochastic setting and the finite-sum setting. Our
theorems indicate that Prox-LEAD works with arbitrary compression precision,
and it tremendously reduces the communication cost almost for free. The
superiorities of the proposed algorithms are demonstrated through the
comparison with state-of-the-art algorithms in terms of convergence
complexities and numerical experiments. Our algorithmic framework also
generally enlightens the compressed communication on other primal-dual
algorithms by reducing the impact of inexact iterations, which might be of
independent interest.
