Communication-Efficient Distributed SGD with Error-Feedback, Revisited
  We show that the convergence proof of a recent algorithm called dist-EF-SGD
for distributed stochastic gradient descent with communication efficiency using
error-feedback of Zheng et al. (NeurIPS 2019) is problematic mathematically.
Concretely, the original error bound for arbitrary sequences of learning rate
is unfortunately incorrect, leading to an invalidated upper bound in the
convergence theorem for the algorithm. As evidences, we explicitly provide
several counter-examples, for both convex and non-convex cases, to show the
incorrectness of the error bound. We fix the issue by providing a new error
bound and its corresponding proof, leading to a new convergence theorem for the
dist-EF-SGD algorithm, and therefore recovering its mathematical analysis.
