Communication-efficient Byzantine-robust distributed learning with
  statistical guarantee
  Communication efficiency and robustness are two major issues in modern
distributed learning framework. This is due to the practical situations where
some computing nodes may have limited communication power or may behave
adversarial behaviors. To address the two issues simultaneously, this paper
develops two communication-efficient and robust distributed learning algorithms
for convex problems. Our motivation is based on surrogate likelihood framework
and the median and trimmed mean operations. Particularly, the proposed
algorithms are provably robust against Byzantine failures, and also achieve
optimal statistical rates for strong convex losses and convex (non-smooth)
penalties. For typical statistical models such as generalized linear models,
our results show that statistical errors dominate optimization errors in finite
iterations. Simulated and real data experiments are conducted to demonstrate
the numerical performance of our algorithms.
