Innovation Compression for Communication-efficient Distributed
  Optimization with Linear Convergence
  Information compression is essential to reduce communication cost in
distributed optimization over peer-to-peer networks. This paper proposes a
communication-efficient linearly convergent distributed (COLD) algorithm to
solve strongly convex optimization problems. By compressing innovation vectors,
which are the differences between decision vectors and their estimates, COLD is
able to achieve linear convergence for a class of $\delta$-contracted
compressors. We explicitly quantify how the compression affects the convergence
rate and show that COLD matches the same rate of its uncompressed version. To
accommodate a wider class of compressors that includes the binary quantizer, we
further design a novel dynamical scaling mechanism and obtain the linearly
convergent Dyna-COLD. Importantly, our results strictly improve existing
results for the quantized consensus problem. Numerical experiments demonstrate
the advantages of both algorithms under different compressors.
