Quantized Decentralized Stochastic Learning over Directed Graphs
  We consider a decentralized stochastic learning problem where data points are
distributed among computing nodes communicating over a directed graph. As the
model size gets large, decentralized learning faces a major bottleneck that is
the heavy communication load due to each node transmitting large messages
(model updates) to its neighbors. To tackle this bottleneck, we propose the
quantized decentralized stochastic learning algorithm over directed graphs that
is based on the push-sum algorithm in decentralized consensus optimization.
More importantly, we prove that our algorithm achieves the same convergence
rates of the decentralized stochastic learning algorithm with
exact-communication for both convex and non-convex losses. Numerical
evaluations corroborate our main theoretical results and illustrate significant
speed-up compared to the exact-communication methods.
