Federated Two-stage Learning with Sign-based Voting
  Federated learning is a distributed machine learning mechanism where local
devices collaboratively train a shared global model under the orchestration of
a central server, while keeping all private data decentralized. In the system,
model parameters and its updates are transmitted instead of raw data, and thus
the communication bottleneck has become a key challenge. Besides, recent larger
and deeper machine learning models also pose more difficulties in deploying
them in a federated environment. In this paper, we design a federated two-stage
learning framework that augments prototypical federated learning with a cut
layer on devices and uses sign-based stochastic gradient descent with the
majority vote method on model updates. Cut layer on devices learns informative
and low-dimension representations of raw data locally, which helps reduce
global model parameters and prevents data leakage. Sign-based SGD with the
majority vote method for model updates also helps alleviate communication
limitations. Empirically, we show that our system is an efficient and privacy
preserving federated learning scheme and suits for general application
scenarios.
